couchdb-couch.git
21 months agoRemove PSE references from compaction test master
Paul J. Davis [Thu, 23 Mar 2017 19:55:48 +0000 (14:55 -0500)] 
Remove PSE references from compaction test

I accidentally included part of the PSE work when I extracted this
patch. Should just be the minor tweak to access the updater pid
directly.

21 months agoFix compaction daemon tests
Paul J. Davis [Thu, 2 Feb 2017 18:33:55 +0000 (12:33 -0600)] 
Fix compaction daemon tests

There was a race condition in the compaction daemon tests where the
compaction daemon would think a compaction had finished but the test
would grab file size info before the compaction had swapped out which
breaks the assertion that fragmentation was reduced.

There was also a slight error in calculating the fragmentation in the
test suite by using the wrong size value.

22 months agoImplement an ETS-basd couch_lru 239/head
Nick Vatamaniuc [Wed, 15 Mar 2017 18:06:25 +0000 (14:06 -0400)] 
Implement an ETS-basd couch_lru

Use a monotonicaly incrementing counter instead of `erlang:now()`. We don't
technically need to time-based functionality and just want to know relative
insertion order.

Instead of gb_tree, use an ordered_set ETS. This keep items sorted by their
update order, with most recent ones at the bottom.

An set ETS replaces the dictionary which maintains a mapping from database
names to their entry in updates table.

Interface is the same as the old couch_lru, so it is a direct swap in.

Thanks to Eric Avdey for intial version of test module.

COUCHDB-3326

22 months agoMerge branch 'couchdb-2992'
Nick Vatamaniuc [Thu, 16 Mar 2017 02:34:25 +0000 (22:34 -0400)] 
Merge branch 'couchdb-2992'

Closes #235

22 months agoAllow limiting maximum document body size
Nick Vatamaniuc [Mon, 13 Mar 2017 06:33:57 +0000 (02:33 -0400)] 
Allow limiting maximum document body size

Configuration is via the `couchdb.max_document_size`. In the past that
was implemented as a maximum http request body size and this finally implements
it by actually checking a document's body size.

COUCHDB-2992

22 months agofix compiler and dialyzer warnings
Robert Newson [Wed, 15 Mar 2017 18:13:09 +0000 (18:13 +0000)] 
fix compiler and dialyzer warnings

22 months agoAdd sys_dbs to the LRU 237/head
Robert Newson [Tue, 14 Mar 2017 17:40:21 +0000 (17:40 +0000)] 
Add sys_dbs to the LRU

COUCHDB-3325

22 months agoMerge remote branch 'cloudant:70794-fix-initial-acc-for-builtin_sum_rows'
ILYA Khlopotov [Wed, 8 Mar 2017 21:22:30 +0000 (13:22 -0800)] 
Merge remote branch 'cloudant:70794-fix-initial-acc-for-builtin_sum_rows'

This closes #234

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
22 months agoFix initial accumulator value for builtin_sum_rows 234/head
ILYA Khlopotov [Wed, 8 Mar 2017 20:24:46 +0000 (12:24 -0800)] 
Fix initial accumulator value for builtin_sum_rows

This a fixup for https://github.com/apache/couchdb-couch/pull/229.
The initial value for accumulator suppose to be 0. Since the result of
sum and count is integer. The problem was discovered by dialyzer.

COUCHDB-3305

22 months agoMerge branch '70794-reduce-sum-errors'
Tony Sun [Wed, 8 Mar 2017 03:41:17 +0000 (19:41 -0800)] 
Merge branch '70794-reduce-sum-errors'

Closes #229

22 months agoReturn error row instead of crashing 229/head
Tony Sun [Tue, 21 Feb 2017 22:18:49 +0000 (14:18 -0800)] 
Return error row instead of crashing

When input is invalid for built in reducers, we return an error row
instead of crashing. Thanks to davisp for providing the elegant
solution.

COUCHDB-3305

22 months agoMerge branch '64229-add-new-request-parameter'
Nick Vatamaniuc [Wed, 8 Mar 2017 01:29:33 +0000 (20:29 -0500)] 
Merge branch '64229-add-new-request-parameter'

Closes #233

22 months agoRename max_document_size to max_http_request_size 233/head
Nick Vatamaniuc [Wed, 8 Mar 2017 00:03:32 +0000 (19:03 -0500)] 
Rename max_document_size to max_http_request_size

`max_document_size` is implemented as `max_http_request_size`. There was no
real check for document size. In some cases the implementation was close enough
of a proxy (PUT-ing and GET-ing single docs), but in some edge cases, like
_bulk_docs requests the discrepancy between request size and document size
could be rather large.

The section was changed accordingly from `couchdb` to `httpd`. `httpd` was
chosen as it applies to both clustered as well as local interface.

There is a parallel effort to implement an actual max_document_size check. The
set of commit should be merged close enough together to allow for a backwards
compatible transition.

COUCHDB-2992

22 months agoMerge remote-tracking branch 'cloudant/3318-bypass-vhosts'
Robert Newson [Mon, 6 Mar 2017 14:06:09 +0000 (14:06 +0000)] 
Merge remote-tracking branch 'cloudant/3318-bypass-vhosts'

22 months agoEfficiently bypass vhost handling if there are none 232/head
Robert Newson [Mon, 6 Mar 2017 12:05:33 +0000 (12:05 +0000)] 
Efficiently bypass vhost handling if there are none

COUCHDB-3318

22 months agoFix `badarith` error in couch_db:get_db_info call 231/head
Nick Vatamaniuc [Fri, 3 Mar 2017 02:24:52 +0000 (21:24 -0500)] 
Fix `badarith` error in couch_db:get_db_info call

When folding we account for a previous `null`, `undefined`, or a number. However
btree:size/1 returns 0, `nil` or a number.

Switched `undefined` to `nil`. Couldn't find where btree:size would ever return
`undefined`, it seems we meant to use `nil` instead.

COUCHDB-3316

22 months agoMerge branch 'COUCHDB-3298-improve-couch-btree-chunkify'
Paul J. Davis [Wed, 1 Mar 2017 21:15:46 +0000 (15:15 -0600)] 
Merge branch 'COUCHDB-3298-improve-couch-btree-chunkify'

22 months agoMerge remote branch 'cloudant:add_sysdb_callback_once_per_db'
ILYA Khlopotov [Tue, 28 Feb 2017 21:51:29 +0000 (13:51 -0800)] 
Merge remote branch 'cloudant:add_sysdb_callback_once_per_db'

This closes #230

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
22 months agoAdd sys db callbacks only once per db open 230/head
brkolla [Sun, 19 Feb 2017 21:33:39 +0000 (16:33 -0500)] 
Add sys db callbacks only once per db open

Currently we are adding the sys db callbacks for every request instead of
once while opening the db. This fix will ensure that we will only add the
sys db call backs once while opening the db.

COUCHDB-3307

23 months agoEnsure multi-item chunks in couch_btree:chunkify/1 COUCHDB-3298-improve-couch-btree-chunkify 228/head
Paul J. Davis [Sat, 11 Feb 2017 21:29:14 +0000 (15:29 -0600)] 
Ensure multi-item chunks in couch_btree:chunkify/1

If the last element of a chunk has a huge reduction it was possible to
return a btree node that had a single key. This prevents the edge case
by forcing it into the previous chunk. Without this we can end up with a
case where a path in the tree can extend for many levels with only a
single key in each node.

COUCHDB-3298

23 months agoMake couch_btree:chunkify/1 prefer fewer chunks
Paul J. Davis [Sat, 11 Feb 2017 21:26:26 +0000 (15:26 -0600)] 
Make couch_btree:chunkify/1 prefer fewer chunks

This changes couch_btree:chunkify/1 to produce fewer larger chunks
rather than creating chunks of even-ish size.

COUCHDB-3298

23 months agoMerge branch 'couchdb-3293'
Nick Vatamaniuc [Thu, 9 Feb 2017 20:36:53 +0000 (15:36 -0500)] 
Merge branch 'couchdb-3293'

Closes #226

23 months agoAllow limiting length of document ID 226/head
Nick Vatamaniuc [Thu, 9 Feb 2017 15:13:42 +0000 (10:13 -0500)] 
Allow limiting length of document ID

Previously it was not possibly to define a maxum document ID size. That meant
large document ID would hit various limitations and corner cases. For example,
large document IDs could be inserted via a _bulk_docs endpoint but then trying
to insert the same document via a single HTTP method like PUT would fail
because of a limitation in Mochiweb's HTTP parser.

Let operators specify a maxium document ID length via the

```
couchdb.max_document_id_length = infinity | Integer
```

configuration.

The default value of `infinity` keeps the current behavior where document
ID length is not checked.

COUCHDB-3293

23 months agoMerge remote branch 'cloudant:add-test_requet_delete_3'
ILYA Khlopotov [Wed, 8 Feb 2017 20:26:29 +0000 (12:26 -0800)] 
Merge remote branch 'cloudant:add-test_requet_delete_3'

This closes #227

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
23 months agorefactor to use greater arity functions 227/head
ILYA Khlopotov [Wed, 8 Feb 2017 20:05:17 +0000 (12:05 -0800)] 
refactor to use greater arity functions

23 months agoAdd test_request:delete/3
ILYA Khlopotov [Wed, 8 Feb 2017 18:47:58 +0000 (10:47 -0800)] 
Add test_request:delete/3

23 months agoMerge branch 'couchdb-3284-remove-dead-code'
Nick Vatamaniuc [Wed, 1 Feb 2017 19:48:07 +0000 (14:48 -0500)] 
Merge branch 'couchdb-3284-remove-dead-code'

Closes #225

23 months agoRemove dead code from couch_file 225/head
Nick Vatamaniuc [Wed, 1 Feb 2017 19:42:59 +0000 (14:42 -0500)] 
Remove dead code from couch_file

This code was left over after removing 8kB read-ahead

https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f

COUCHDB-3284

23 months agoMerge branch 'couchdb-3284'
Nick Vatamaniuc [Tue, 31 Jan 2017 15:52:47 +0000 (10:52 -0500)] 
Merge branch 'couchdb-3284'

closes #223

23 months agoMerge remote branch 'cloudant:82197-fix-validation_func-case_clause'
Eric Avdey [Tue, 31 Jan 2017 13:27:28 +0000 (09:27 -0400)] 
Merge remote branch 'cloudant:82197-fix-validation_func-case_clause'

This closes #224

Signed-off-by: Eric Avdey <eiri@eiri.ca>
23 months agoDon't crash on unexpected validation's error type 224/head
Eric Avdey [Mon, 30 Jan 2017 19:27:59 +0000 (15:27 -0400)] 
Don't crash on unexpected validation's error type

Validation function can throw JSON with error
type different than forbidden or unauthorized.

This patch makes couch return unknown_error
instead of crashing query server.

COUCHDB-3286

23 months agoRemove 8kB read-ahead from couch_file 223/head
Nick Vatamaniuc [Fri, 27 Jan 2017 17:04:01 +0000 (12:04 -0500)] 
Remove 8kB read-ahead from couch_file

In production it showed an increased input Erlang IO and increased binary
memory usage.

See attached file in Jira ticket:

COUCHDB-3284

23 months agoReset EOF if a partial write was possible 222/head
Robert Newson [Thu, 19 Jan 2017 23:12:00 +0000 (23:12 +0000)] 
Reset EOF if a partial write was possible

We can't know if one or more bytes were written by a file:write/2 call
that results in an error and so it is not correct to leave #file.eof
at its original value. In the event of error, use file:position(Fd,
eof) to find the new, true length of the file, and update #file{}
accordingly.

COUCHDB-3274

23 months agoMerge remote branch 'cloudant:validate-count-in-uuids'
Eric Avdey [Wed, 18 Jan 2017 15:34:00 +0000 (11:34 -0400)] 
Merge remote branch 'cloudant:validate-count-in-uuids'

This closes #221

Signed-off-by: Eric Avdey <eiri@eiri.ca>
23 months agoReturn "Bad Request" when count in /_uuids exceeds max 221/head
Eric Avdey [Wed, 18 Jan 2017 13:55:19 +0000 (09:55 -0400)] 
Return "Bad Request" when count in /_uuids exceeds max

23 months agoValidate that count in /_uuids is positive
Eric Avdey [Wed, 18 Jan 2017 13:44:15 +0000 (09:44 -0400)] 
Validate that count in /_uuids is positive

2 years agoMerge remote branch 'cloudant:fix-views_tests'
Eric Avdey [Fri, 6 Jan 2017 14:06:44 +0000 (10:06 -0400)] 
Merge remote branch 'cloudant:fix-views_tests'

This closes #220

Signed-off-by: Eric Avdey <eiri@eiri.ca>
2 years agoFix couchdb_views tests 220/head
Eric Avdey [Thu, 5 Jan 2017 21:30:00 +0000 (17:30 -0400)] 
Fix couchdb_views tests

Previous commit made couch_file to monitor db_pid
instead of linking ot it, so number of monitored_by
for couch_db process, checked in couchdb_views_tests,
changed.

COUCHDB-3259

2 years agoDon't trap exits in couch_file 219/head
Robert Newson [Tue, 20 Dec 2016 22:32:44 +0000 (22:32 +0000)] 
Don't trap exits in couch_file

It turns out to be impossible to trap the exit message from the parent
process in a gen_server, which is the only reason couch_file traps
exits in the first place (i.e, this has never worked).

This commit also changes the hot upgrade code to handle this
transition (but not the previous one).

COUCHDB-3259

2 years agoMerge branch 'COUCHDB-3255-fix-new-revid'
Paul J. Davis [Thu, 15 Dec 2016 15:47:27 +0000 (09:47 -0600)] 
Merge branch 'COUCHDB-3255-fix-new-revid'

2 years agoMake revision generation deterministic 218/head
Paul J. Davis [Tue, 13 Dec 2016 23:30:46 +0000 (17:30 -0600)] 
Make revision generation deterministic

This removes the influence of the attachment disk information when
generating revisions when a document is being recreated (ie, it existed
once but has been deleted).

Big thanks to @tonysun83 for getting a reproducer as well as the test
included with this PR.

COUCHDB-3255

2 years agoMerge remote branch 'cloudant:add-filter-changes-test'
Eric Avdey [Tue, 13 Dec 2016 14:07:48 +0000 (10:07 -0400)] 
Merge remote branch 'cloudant:add-filter-changes-test'

This closes #213

Signed-off-by: Eric Avdey <eiri@eiri.ca>
2 years agoAdd tests for filter by filter function 213/head
Eric Avdey [Sat, 12 Nov 2016 00:50:37 +0000 (20:50 -0400)] 
Add tests for filter by filter function

2 years agoReduce duplicated code, rename vars consistently
Eric Avdey [Sat, 12 Nov 2016 00:38:50 +0000 (20:38 -0400)] 
Reduce duplicated code, rename vars consistently

2 years agoPrevent consumer from hiding error messages
Eric Avdey [Fri, 11 Nov 2016 22:53:23 +0000 (18:53 -0400)] 
Prevent consumer from hiding error messages

2 years agoMerge branch '3251-remove-filename-rootname'
Paul J. Davis [Wed, 7 Dec 2016 17:10:36 +0000 (11:10 -0600)] 
Merge branch '3251-remove-filename-rootname'

2 years agoRemove use of filename:rootname/1 217/head
Paul J. Davis [Tue, 6 Dec 2016 19:37:45 +0000 (13:37 -0600)] 
Remove use of filename:rootname/1

It turns out that filename:rootname/1 is fairly expensive. Given that we
call it millions of times when doing database name validation it adds up
significantly. This is a fairly easy optimization to just use binary
matching instead.

COUCHDB-3251

2 years agoMake couchjs -S option take effect 216/head
Nick Vatamaniuc [Wed, 30 Nov 2016 17:26:56 +0000 (12:26 -0500)] 
Make couchjs -S option take effect

Previously it was used to set JS context's stack chunk size.

Instead, to be effective it should set max GC size of the runtime.

Stack chunk size was set to the recommended value: 8K

This brings back an accidentally reverted commit:

https://github.com/apache/couchdb-couch/commit/62dafe81e13d7f8e27e95057e65f76d534aa2313

by @tilgovi

Reference:

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS_NewContext

COUCHDB-3245

2 years agoHandle open_result message that arrives after the delete 214/head
Robert Newson [Wed, 23 Nov 2016 15:20:11 +0000 (15:20 +0000)] 
Handle open_result message that arrives after the delete

Opening a database is asynchronous, therefore it's possible for the
database to have been deleted before the open is complete. The
open_result code assumes this doesn't happen, so when it does,
couch_server crashes, taking the couch_dbs ETS table with it.

Explicitly handle the case and keep on keeping on.

COUCHDB-3241

2 years agoMerge remote branch 'cloudant:71810-handle-errors-terms-from-fabric'
ILYA Khlopotov [Sat, 12 Nov 2016 00:22:24 +0000 (16:22 -0800)] 
Merge remote branch 'cloudant:71810-handle-errors-terms-from-fabric'

This closes #206

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoAdd test suite for handling errors from fabric 206/head
ILYA Khlopotov [Thu, 13 Oct 2016 21:56:19 +0000 (14:56 -0700)] 
Add test suite for handling errors from fabric

COUCHDB-3195

2 years agoMerge remote branch 'cloudant:78142-allow-user-to-correct-invalid-user-docs'
ILYA Khlopotov [Fri, 11 Nov 2016 14:34:25 +0000 (06:34 -0800)] 
Merge remote branch 'cloudant:78142-allow-user-to-correct-invalid-user-docs'

This closes #211

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoAllow fixing users' documents 211/head
ILYA Khlopotov [Thu, 10 Nov 2016 23:02:51 +0000 (15:02 -0800)] 
Allow fixing users' documents

Allow adding missing fields:
- type
- roles

Allow update of the type field from incorrect value to the correct one
(currently it suppose to have value 'user').

COUCHDB-3231

2 years agoMerge remote branch 'cloudant:75431-truncate-revs-to-revs_limit'
ILYA Khlopotov [Thu, 10 Nov 2016 17:51:47 +0000 (09:51 -0800)] 
Merge remote branch 'cloudant:75431-truncate-revs-to-revs_limit'

This closes #205

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoTruncate revs returned from couch_db:open_doc_revs 205/head
ILYA Khlopotov [Thu, 13 Oct 2016 17:27:25 +0000 (10:27 -0700)] 
Truncate revs returned from couch_db:open_doc_revs

Return max revs_limit revisions from couch_db:open_doc_revs.

COUCHDB-3193

2 years agoAdd optional `fields` to change feed selectors 177/head
Nick Vatamaniuc [Thu, 2 Jun 2016 21:04:17 +0000 (17:04 -0400)] 
Add optional `fields` to change feed selectors

When using selectors with `include_docs=true` can specify an optional fields
array in the POST request JSON body.

Each element in the array can be a json field (or even a key path
specified as field1.field2...). Resulting documents will contain only the
specified document fields.

For example:
`
http://.../d1/_changes?filter=_selector&include_docs=true

{
  "selector": {"z" : {"$gte" : 1} }, "fields": ["field1", "field2"]

}
`
Will first select only document with "z" value >= 1, then will return only field1 and field2 in documents.

{ "field1": "field1value", "field2": "field2value"}

(This requires a companion pr in fabric to work)

Jira: COUCHDB-2988

2 years agoMerge remote branch 'cloudant:log_filepath_on_errors'
ILYA Khlopotov [Mon, 7 Nov 2016 18:15:02 +0000 (10:15 -0800)] 
Merge remote branch 'cloudant:log_filepath_on_errors'

This closes #210

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoInclude file_path on errors from couch_file 210/head
ILYA Khlopotov [Thu, 3 Nov 2016 16:05:00 +0000 (09:05 -0700)] 
Include file_path on errors from couch_file

Add PathFile as part of a reason for following errors:
 - read_beyond_eof
 - exceed_pread_limit exceptions

2 years agoMerge remote branch 'cloudant:add-couch_debug-opened_files_by_regexp'
ILYA Khlopotov [Fri, 21 Oct 2016 19:09:04 +0000 (12:09 -0700)] 
Merge remote branch 'cloudant:add-couch_debug-opened_files_by_regexp'

This closes #207

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoAdd couch_debug:opened_files_contains/1 207/head
ILYA Khlopotov [Fri, 21 Oct 2016 18:51:05 +0000 (11:51 -0700)] 
Add couch_debug:opened_files_contains/1

couch_debug:opened_files_contains is an efficient way to find out list
of file descriptors or couch_file processes which path contains given
substring.

2 years agoAdd couch_debug:opened_files_by_regexp/1
ILYA Khlopotov [Fri, 21 Oct 2016 18:26:18 +0000 (11:26 -0700)] 
Add couch_debug:opened_files_by_regexp/1

couch_debug:opened_files_by_regexp is an efficient way to find out list
of file descriptors or couch_file processes which path matches given
regexp

2 years agowhitespace change to trigger github mirror sync
Robert Newson [Wed, 12 Oct 2016 20:54:00 +0000 (21:54 +0100)] 
whitespace change to trigger github mirror sync

2 years agoFix module name in couch_httpd:check_max_request_length/1 call 204/head
Nick Vatamaniuc [Mon, 10 Oct 2016 05:47:49 +0000 (01:47 -0400)] 
Fix module name in couch_httpd:check_max_request_length/1 call

2 years agoMerge branch 'couchdb-3174'
Nick Vatamaniuc [Thu, 6 Oct 2016 21:14:52 +0000 (17:14 -0400)] 
Merge branch 'couchdb-3174'

2 years agoMerge remote-tracking branch 'banjiewen/stale-stable-update'
Robert Newson [Thu, 6 Oct 2016 18:01:50 +0000 (19:01 +0100)] 
Merge remote-tracking branch 'banjiewen/stale-stable-update'

2 years agoMerge remote branch 'cloudant:fix-find_newest_header-type-spec'
ILYA Khlopotov [Wed, 5 Oct 2016 21:28:51 +0000 (14:28 -0700)] 
Merge remote branch 'cloudant:fix-find_newest_header-type-spec'

This closes #203

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoupdate typespec of find_newest_header 203/head
ILYA Khlopotov [Wed, 5 Oct 2016 21:25:19 +0000 (14:25 -0700)] 
update typespec of find_newest_header

2 years agoMerge remote branch 'cloudant:69425-handle-truncated-req-in-recv_body'
ILYA Khlopotov [Wed, 5 Oct 2016 17:01:26 +0000 (10:01 -0700)] 
Merge remote branch 'cloudant:69425-handle-truncated-req-in-recv_body'

This closes #200

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoAdding test suite for trancated _update 200/head
ILYA Khlopotov [Thu, 22 Sep 2016 23:11:02 +0000 (16:11 -0700)] 
Adding test suite for trancated _update

COUCHDB-3158

2 years agoMerge remote-tracking branch 'cloudant/3061-adaptive-header-search'
Paul J. Davis [Tue, 4 Oct 2016 18:18:35 +0000 (13:18 -0500)] 
Merge remote-tracking branch 'cloudant/3061-adaptive-header-search'

2 years agoUse vectored reads to search for buried headers 185/head
Jay Doane [Tue, 4 Oct 2016 17:58:51 +0000 (10:58 -0700)] 
Use vectored reads to search for buried headers

Current behavior attempts to read a header at each block, starting at
the eof and working backwards one block at a time. Deeply buried headers
can take a long time to find, depending on the depth of the header,
server load, etc.

This commit changes the behavior so that if the last block in the file
does not contain a header, it switches to using a "vectored" approach
where multiple candidate header prefixes (of 5 bytes) are read in a
single operation, greatly speeding up the search. On a modern linux
system with SSD, we see improvements up to 15x.

COUCHDB-3061

2 years agoMerge branch '3173-fix-couchjs-character-replacement'
Paul J. Davis [Tue, 4 Oct 2016 14:51:43 +0000 (09:51 -0500)] 
Merge branch '3173-fix-couchjs-character-replacement'

2 years agoFix CouchJS character replacement 202/head
Paul J. Davis [Tue, 4 Oct 2016 14:45:36 +0000 (09:45 -0500)] 
Fix CouchJS character replacement

This was a bad backport from an old bug. We accidentally backed up when
looking at the second half of a surrogate pair. Instead the backup
should only happen when we see a low half of a surrogate pair with no
preceding high half.

COUCHDB-3173

2 years agoAdd max_document_size checking for multipart PUT requests 201/head
Nick Vatamaniuc [Tue, 4 Oct 2016 06:50:07 +0000 (02:50 -0400)] 
Add max_document_size checking for multipart PUT requests

Previously multipart/related PUT requests didn't check maximum request sizes.

This commit checks content-length and compares that with the maximum.

This means keeping the current "semantics" of max_document_size which actually
means "max request size". But this makes the check more efficient and can
be done earlier in request processing time.

Jira: COUCHDB-3174

2 years agoMerge remote branch 'cloudant:74046-authentication_db-should-be-sysdb' test-branch
ILYA Khlopotov [Tue, 20 Sep 2016 19:51:07 +0000 (12:51 -0700)] 
Merge remote branch 'cloudant:74046-authentication_db-should-be-sysdb'

This closes #198

Signed-off-by: ILYA Khlopotov <iilyak@apache.org>
2 years agoMake sure clustered authentication_db is sys_db 198/head
ILYA Khlopotov [Tue, 13 Sep 2016 18:42:47 +0000 (11:42 -0700)] 
Make sure clustered authentication_db is sys_db

Fixes the case when the following are configured to something other than _users
 - chttpd_auth.authentication_db
 - couch_httpd_auth.authentication_db

Introduce "couchdb.users_db_suffix" config key

Remove distinction between clustered and local users databases
configuration. We remove following configuration options and replace
them with "couchdb.users_db_suffix":

- chttpd_auth.authentication_db
- couch_httpd_auth.authentication_db

COUCHDB-2731

2 years agoMerge remote branch 'cloudant:define_couch_version'
ILYA Khlopotov [Tue, 20 Sep 2016 17:03:36 +0000 (10:03 -0700)] 
Merge remote branch 'cloudant:define_couch_version'

This closes #199

Signed-off-by: ILYA Khlopotov <iilyak@ca.ibm.com>
2 years agoDefine COUCHDB_VERSION macro 199/head
ILYA Khlopotov [Fri, 16 Sep 2016 20:13:08 +0000 (13:13 -0700)] 
Define COUCHDB_VERSION macro

2 years agoRemove rebar.config
ILYA Khlopotov [Fri, 16 Sep 2016 20:11:56 +0000 (13:11 -0700)] 
Remove rebar.config

2 years agoRevert "Get couch version from environment not git"
ILYA Khlopotov [Fri, 16 Sep 2016 20:38:53 +0000 (13:38 -0700)] 
Revert "Get couch version from environment not git"

This reverts commit c613dabf635ca3a5d6de2240d00b80c334dc8f43.

2 years agoRevert "get version from environment in cross-platform way"
ILYA Khlopotov [Fri, 16 Sep 2016 20:38:40 +0000 (13:38 -0700)] 
Revert "get version from environment in cross-platform way"

This reverts commit 7d0cfb5f9d1f0fcdc607b46f31ae33cbf65115df.

2 years agoPartial revert Fix Windows version string determ.."
ILYA Khlopotov [Fri, 16 Sep 2016 20:36:34 +0000 (13:36 -0700)] 
Partial revert Fix Windows version string determ.."

This partially reverts commit 70b61f08618c34f4604bebc2ad2b8d71f90cca5e.

2 years agoMerge branch 'fix-windows-version' of https://github.com/wohali/couchdb-couch
Joan Touzet [Mon, 12 Sep 2016 19:52:51 +0000 (15:52 -0400)] 
Merge branch 'fix-windows-version' of https://github.com/wohali/couchdb-couch

2 years agoFix Windows version string determination 197/head
Joan Touzet [Mon, 12 Sep 2016 19:28:31 +0000 (15:28 -0400)] 
Fix Windows version string determination

For some reason, the COUCHD_VERSION string on Windows is getting
suffixed whitespace. We fix this by string:strip(Version0, right) on the
variable before use.

2 years agoget version from environment in cross-platform way
Robert Newson [Mon, 12 Sep 2016 09:28:51 +0000 (10:28 +0100)] 
get version from environment in cross-platform way

2 years agoGet couch version from environment not git
Robert Newson [Sun, 11 Sep 2016 11:47:51 +0000 (12:47 +0100)] 
Get couch version from environment not git

2 years agoMerge branch '3140-disable-queue-as-fifo-test' of https://github.com/wohali/couchdb...
Joan Touzet [Sun, 11 Sep 2016 08:54:27 +0000 (04:54 -0400)] 
Merge branch '3140-disable-queue-as-fifo-test' of https://github.com/wohali/couchdb-couch

2 years agoDisable should_process_waiting_queue_as_fifo test 196/head
Joan Touzet [Sun, 11 Sep 2016 06:24:40 +0000 (02:24 -0400)] 
Disable should_process_waiting_queue_as_fifo test

COUCHDB-3140

This PR disables the problematic queue-as-fifo test. As written, any
call in a test to spawn_client/1 returns immediately, and does not
guarantee that the call to couch_query_servers:get_ddoc_process/2
returns before control flow proceeds to the next line in the eunit test.
As it turns out, on Windows, the call to spawn_client for ddoc5 succeeds
prior to the ddoc4 call in this test, thus always failing.

Insertion of a timer:sleep/1 call between the spawn_client/1 calls seems
to solve the problem, but even this is "lucky" behaviour. Semantically
the way this test is written, with spawn_client/1 using spawn/1 to spin
off the actual get_ddoc_process/2 call, there is no guarantee that the
couch_proc_manager FIFO queue is being populated correctly.

Further, if the get_ddoc_process/2 call takes longer than the defined
timeout (5000 ms) this test will always fail, as no provision is made to
keep trying for an os_process after that time. Again we are "lucky" that
the test runs fast enough that this is not a problem, but relying on a
performant BEAM VM for test success in any regard is equally
problematic.

For this reason I am completely commenting out this test until the
harness can be fixed. I believe the impact to couchdb is minimal.

(It also suggests we may want to rethink how requests for os processes
are queued and issued at some point in the future; this approach seems
sub-optimal, especially if strict FIFO ordering is an expectation of the
system.)

/cc @eiri @rnewson @janl

2 years agoFix unused variables warning
Robert Newson [Sat, 10 Sep 2016 09:03:54 +0000 (10:03 +0100)] 
Fix unused variables warning

UserCtx was available here in the past so we could upgrade password
hashes when authenticating. In the clustered case we found this to be
messy as multiple attempts to upgrade the same document would occur
close together. For 2.0, we elected to remove this upgrade-on-auth
path.

2 years agooption to disable runtime code evaluation
Randall Leeds [Sun, 24 Apr 2016 00:49:34 +0000 (17:49 -0700)] 
option to disable runtime code evaluation

2 years agoMerge remote branch 'cloudant:3114-fix-docid-validation'
Eric Avdey [Thu, 1 Sep 2016 14:10:27 +0000 (11:10 -0300)] 
Merge remote branch 'cloudant:3114-fix-docid-validation'

This closes #193

Signed-off-by: Eric Avdey <eiri@eiri.ca>
2 years agoFix validation for ddoc name 193/head
Eric Avdey [Thu, 1 Sep 2016 12:51:14 +0000 (09:51 -0300)] 
Fix validation for ddoc name

2 years agofeat: allow couchspawnkillable to live in directories with spaces
Jan Lehnardt [Wed, 31 Aug 2016 19:39:13 +0000 (21:39 +0200)] 
feat: allow couchspawnkillable to live in directories with spaces

2 years agoMerge remote branch 'cloudant:3102-fix-config_subscription'
ILYA Khlopotov [Tue, 23 Aug 2016 21:59:27 +0000 (14:59 -0700)] 
Merge remote branch 'cloudant:3102-fix-config_subscription'

This closes #192

Signed-off-by: ILYA Khlopotov <iilyak@ca.ibm.com>
2 years agoUse `define` to avoid repeating yourself 192/head
ILYA Khlopotov [Tue, 23 Aug 2016 20:15:57 +0000 (13:15 -0700)] 
Use `define` to avoid repeating yourself

COUCHDB-3102

2 years agoUpdate handle_config_terminate API
ILYA Khlopotov [Wed, 17 Aug 2016 18:24:08 +0000 (11:24 -0700)] 
Update handle_config_terminate API

COUCHDB-3102

2 years agoMerge remote branch 'cloudant:66640-simplify-proc_manager'
Eric Avdey [Tue, 16 Aug 2016 15:08:56 +0000 (12:08 -0300)] 
Merge remote branch 'cloudant:66640-simplify-proc_manager'

This closes #191

Signed-off-by: Eric Avdey <eiri@eiri.ca>
2 years agoAdd tests for process pool 191/head
Eric Avdey [Mon, 15 Aug 2016 16:45:20 +0000 (13:45 -0300)] 
Add tests for process pool

This adds tests for re-use of procs,
proper waiters dequeue and idle pool reduction

2 years agoSimplify proc manager assignment logic
Eric Avdey [Thu, 11 Aug 2016 20:45:32 +0000 (17:45 -0300)] 
Simplify proc manager assignment logic

This changes how proc manager handles proc assignment.
Instead of doing this in three different places:
get_proc call handler, return_proc/2 with maybe_assign_proc
and flush_waiters/2, proc manager now just places
all the incoming requests in the waiting queue
and then flushes it.

As a result all the logic kept in one place which makes
it more obvious that we are treating proc management
as a processing of a single FIFO queue
with "soft" and "hard" upper limits.

Consequently this is fixing a bug in maybe_assign_proc
where it was possible to assign a client a process
that wasn't aware of it.

COUCHDB-3095

2 years agoMerge remote branch 'cloudant:3080-fix-validate_dbname'
ILYA Khlopotov [Wed, 10 Aug 2016 19:35:58 +0000 (12:35 -0700)] 
Merge remote branch 'cloudant:3080-fix-validate_dbname'

This closes #188

Signed-off-by: ILYA Khlopotov <iilyak@ca.ibm.com>