couchdb-ioq.git
7 weeks agoMerge pull request #18 from apache/guard-negative-timedelta main
Jay Doane [Wed, 6 Apr 2022 18:02:01 +0000 (11:02 -0700)] 
Merge pull request #18 from apache/guard-negative-timedelta

IOQ improvements

7 weeks agoHandle stats db conflicts guard-negative-timedelta 18/head
Jay Doane [Tue, 5 Apr 2022 23:58:03 +0000 (16:58 -0700)] 
Handle stats db conflicts

Since a timestamp is used as the document id, it's possible for e.g.
multiple nodes to attempt to insert the same document, resulting in
logged errors like the following:

[error] 2022-04-05T21:00:35.302496Z node1@127.0.0.1 emulator -------- Error in process <0.588.159> on node 'node1@127.0.0.1' with exit value:
{{nocatch,conflict},[{fabric,update_doc,3,[{file,"src/fabric.erl"},{line,332}]},{ioq_server,save_to_db,0,[{file,"src/ioq_server.erl"},{line,529}]}]}

This catches cases of `error:conflict` and logs them at info level
instead.

7 weeks agoGuard against negative time deltas
Jay Doane [Tue, 5 Apr 2022 19:48:10 +0000 (12:48 -0700)] 
Guard against negative time deltas

Despite the use of `erlang:monotonic_time/0`, it is still possible on
some platforms, such as sleeping/awakening laptops, for time to move
in surprising directions. When this happens, and a negative time delta
is passed to `timebin/1`, it crashes with:

bad arithmetic expression at math:log10/1 <= ioq_server:timebin/1(line:480)

This adds a guard to return `0` for negative numbers as well as zero.

6 months agoMerge pull request #17 from apache/app-env-to-config
Jay Doane [Wed, 3 Nov 2021 06:06:10 +0000 (23:06 -0700)] 
Merge pull request #17 from apache/app-env-to-config

Replace app env with config

6 months agoReplace app env with config 17/head
Jay Doane [Tue, 2 Nov 2021 21:12:26 +0000 (14:12 -0700)] 
Replace app env with config

Currently, `stats_db` and `interval` are obtained from vm.args file
via `application:get_env/2`. This is surprising since IOQ already
depends on the config application for most of it's configuration
parameters.

This changes where those configuration values are obtained from
vm.args to .ini files.

2 years agoReduce logging noise master 2.1.3
Adam Kocoloski [Fri, 22 Nov 2019 03:57:25 +0000 (19:57 -0800)] 
Reduce logging noise

2 years agoMerge pull request #15 from apache/ioq_update_hist 2.1.2
iilyak [Wed, 26 Jun 2019 20:09:00 +0000 (13:09 -0700)] 
Merge pull request #15 from apache/ioq_update_hist

Log from 0 is undefined

2 years agoLog from 0 is undefined 15/head
ILYA Khlopotov [Wed, 26 Jun 2019 18:53:34 +0000 (18:53 +0000)] 
Log from 0 is undefined

3 years agoMerge pull request #13 from apache/allow-for-dynamic-ioq-classes
Russell Branca [Thu, 18 Apr 2019 21:23:09 +0000 (14:23 -0700)] 
Merge pull request #13 from apache/allow-for-dynamic-ioq-classes

Allow for dynamic ioq classes

3 years agoAdd dynamic IOQ class documentation allow-for-dynamic-ioq-classes 13/head
Russell Branca [Thu, 18 Apr 2019 21:18:28 +0000 (21:18 +0000)] 
Add dynamic IOQ class documentation

3 years agoAllow dynamic classes in is_valid_class
Russell Branca [Fri, 12 Apr 2019 22:36:18 +0000 (22:36 +0000)] 
Allow dynamic classes in is_valid_class

This adds support for dynamic classes in IOQ2. Before this change, the
allowed set of classes was fixed and hardcoded, with no way for 3rd
party IO traffic to be prioritized, and similarly, no way for 3rd party
IO traffic to bypass IOQ based on config settings. This change switches
to checking to see if there's a declared IO priority value in the
classes config, and if so it is treated as a valid value. It's worth
noting that because `ioq_config:set_class_config` uses the
`is_valid_class` check, you must use `config:set` directly to declare an
initial priority for the dynamic channel before being able to use the
`ioq_config` helpers.

This change was motivated by not being able to bypass search in the
Dreyfus project, as it piggy-backed off of `interactive` and
`view_update` channels. That was rectified in [1], and now Dreyfus uses
the `search` IOQ channel. As that is not used in CouchDB directly, this
commit adds the dynamic class support based off of config values, rather
than specifically hardcoding search, so that it can be used by any 3rd
party services running IO traffic through IOQ.

[1] https://github.com/cloudant-labs/dreyfus/pull/46/commits/75d86c49c86f4e7e299d2d05a741c4ef68a144fb

3 years agoRearrange IOQ config declarations into header file
Russell Branca [Fri, 12 Apr 2019 22:35:55 +0000 (22:35 +0000)] 
Rearrange IOQ config declarations into header file

3 years agoMerge pull request #12 from cloudant/start-couch_log-in-test 2.1.1
iilyak [Mon, 18 Mar 2019 09:28:31 +0000 (02:28 -0700)] 
Merge pull request #12 from cloudant/start-couch_log-in-test

Start couch_log in tests

3 years agoStart couch_log in tests 12/head
ILYA Khlopotov [Fri, 15 Mar 2019 18:14:48 +0000 (18:14 +0000)] 
Start couch_log in tests

3 years agoMerge pull request #11 from apache/optional-proper 2.1.0
Robert Newson [Thu, 28 Feb 2019 19:00:02 +0000 (19:00 +0000)] 
Merge pull request #11 from apache/optional-proper

Make PropEr an optional (test) dependency

3 years agoMake PropEr an optional (test) dependency 11/head
Robert Newson [Thu, 28 Feb 2019 14:07:13 +0000 (14:07 +0000)] 
Make PropEr an optional (test) dependency

3 years agoMerge pull request #10 from cloudant/fix-compilation-warnings
iilyak [Thu, 21 Feb 2019 17:15:37 +0000 (09:15 -0800)] 
Merge pull request #10 from cloudant/fix-compilation-warnings

Fix compiler warnings

3 years agoFix compiler warnings 10/head
ILYA Khlopotov [Wed, 20 Feb 2019 19:16:14 +0000 (19:16 +0000)] 
Fix compiler warnings

- Replace use of `erlang:now/0` with `erlang:monotonic_time/0`
- Replace use of `random` module with `rand` module

3 years agoMerge pull request #9 from cloudant/use-cast-for-ioq2-update_config
iilyak [Thu, 21 Feb 2019 11:37:58 +0000 (03:37 -0800)] 
Merge pull request #9 from cloudant/use-cast-for-ioq2-update_config

Use cast to send `update_config` to `ioq2`

3 years agoUse cast to send update_config to ioq2 9/head
ILYA Khlopotov [Tue, 19 Feb 2019 20:11:23 +0000 (20:11 +0000)] 
Use cast to send update_config to ioq2

3 years agoMerge pull request #8 from cloudant/update_handle_config_terminate
iilyak [Tue, 19 Feb 2019 19:37:20 +0000 (11:37 -0800)] 
Merge pull request #8 from cloudant/update_handle_config_terminate

Reconfigure IOQ on config update

3 years agoReconfigure IOQ on config update 8/head
ILYA Khlopotov [Fri, 15 Feb 2019 13:01:22 +0000 (13:01 +0000)] 
Reconfigure IOQ on config update

3 years agoMerge pull request #7 from apache/cloudant-ioq 2.0.0
Robert Newson [Wed, 30 Jan 2019 17:08:56 +0000 (17:08 +0000)] 
Merge pull request #7 from apache/cloudant-ioq

Import Cloudant's full IOQ code

3 years agoImport Cloudant's full IOQ code 7/head
Robert Newson [Wed, 30 Jan 2019 12:17:08 +0000 (12:17 +0000)] 
Import Cloudant's full IOQ code

4 years agoMerge branch 'use-couch-rand-module' 1.0.1
Nick Vatamaniuc [Thu, 5 Oct 2017 17:22:37 +0000 (13:22 -0400)] 
Merge branch 'use-couch-rand-module'

Fixes #6

4 years agoUse couch_rand compatibility module 6/head
Nick Vatamaniuc [Thu, 5 Oct 2017 16:05:28 +0000 (12:05 -0400)] 
Use couch_rand compatibility module

5 years agoMerge remote branch 'cloudant:remove-unused-config-subscriber' 1.0.0
Eric Avdey [Tue, 4 Oct 2016 16:10:57 +0000 (13:10 -0300)] 
Merge remote branch 'cloudant:remove-unused-config-subscriber'

This closes #5

Signed-off-by: Eric Avdey <eiri@eiri.ca>
5 years agoRemove unused code 5/head
Eric Avdey [Mon, 3 Oct 2016 14:42:02 +0000 (11:42 -0300)] 
Remove unused code

We are subscribing both ioq and ioq_sup to config_event,
but while ioq is actually processing config changes,
ioq_sup just sends uncatched messages to unexisting ioq_server.

5 years agoMerge remote branch 'cloudant:3102-fix-config_subscription'
ILYA Khlopotov [Tue, 23 Aug 2016 21:59:51 +0000 (14:59 -0700)] 
Merge remote branch 'cloudant:3102-fix-config_subscription'

This closes #4

Signed-off-by: ILYA Khlopotov <iilyak@ca.ibm.com>
5 years agoFix a typo in a child name 4/head
ILYA Khlopotov [Tue, 23 Aug 2016 21:23:24 +0000 (14:23 -0700)] 
Fix a typo in a child name

COUCHDB-3102

5 years agoUpdate handle_config_terminate API
ILYA Khlopotov [Mon, 22 Aug 2016 21:42:07 +0000 (14:42 -0700)] 
Update handle_config_terminate API

COUCHDB-3102

7 years agoMerge remote-tracking branch 'iilyak/2561-make-config-API-consistent'
Alexander Shorin [Wed, 4 Feb 2015 15:43:21 +0000 (18:43 +0300)] 
Merge remote-tracking branch 'iilyak/2561-make-config-API-consistent'

This closes #3

COUCHDB-2561

7 years agoDon't restart event handler on termination 3/head
ILYA Khlopotov [Fri, 30 Jan 2015 19:30:02 +0000 (11:30 -0800)] 
Don't restart event handler on termination

COUCHDB-2561

7 years agoUpdate config_listener behaviuor
ILYA Khlopotov [Fri, 30 Jan 2015 18:59:43 +0000 (10:59 -0800)] 
Update config_listener behaviuor

COUCHDB-2561

7 years agoHandle {gen_event_EXIT,{config_listener,ioq},shutdown} message
Alexander Shorin [Tue, 13 Jan 2015 00:28:39 +0000 (03:28 +0300)] 
Handle {gen_event_EXIT,{config_listener,ioq},shutdown} message

7 years agoAllow to customize concurrency value
Alexander Shorin [Mon, 22 Sep 2014 13:35:37 +0000 (17:35 +0400)] 
Allow to customize concurrency value

7 years agoUpdate state on config changes
Alexander Shorin [Mon, 22 Sep 2014 13:32:44 +0000 (17:32 +0400)] 
Update state on config changes

7 years agoFallback to direct I/O if ioq is not running
Robert Newson [Fri, 5 Sep 2014 17:22:44 +0000 (18:22 +0100)] 
Fallback to direct I/O if ioq is not running

7 years agonew IOQ api
Robert Newson [Mon, 1 Sep 2014 18:20:09 +0000 (19:20 +0100)] 
new IOQ api

7 years agoInitial commit
Robert Newson [Thu, 17 Jul 2014 16:53:45 +0000 (17:53 +0100)] 
Initial commit

This is substantively the work from branch 1775-feature-io-regulator
but with erlang application paraphenalia.