geode-benchmarks.git
4 days agoGEODE-10089: Update benchmark baseline develop
Owen Nichols [Wed, 22 Jun 2022 19:15:01 +0000 (12:15 -0700)] 
GEODE-10089: Update benchmark baseline

Updates the default benchmark baseline on develop to 1.15.0

4 weeks agoAdds Java 17 support. (#169)
Jacob Barrett [Thu, 26 May 2022 17:34:22 +0000 (10:34 -0700)] 
Adds Java 17 support. (#169)

* Removes some untested JVM options leftover from Redis benchmarking.
* Adds per-role heap and GC project configurations.
* Extends maxKey option to all benchmarks.
* Uses Gradle Toolchain feature.

4 weeks agoGEODE-10296: Prevent JVM exit on parser error. (#168)
Jacob Barrett [Tue, 24 May 2022 16:29:11 +0000 (09:29 -0700)] 
GEODE-10296: Prevent JVM exit on parser error. (#168)

Configure HistogramLogProcessor to throw exceptions rather than exit.

7 weeks agoGEODE-10278: Remove the geode-for-redis benchmarks
Dan Smith [Wed, 4 May 2022 21:49:28 +0000 (14:49 -0700)] 
GEODE-10278: Remove the geode-for-redis benchmarks

This module is being removed from the geode repository, so we need to remove
the corresponding benchmarks as well.

2 months agoGEODE-10126: Use system properties to configure redis
Dan Smith [Wed, 6 Apr 2022 16:05:04 +0000 (09:05 -0700)] 
GEODE-10126: Use system properties to configure redis

Use system properties to configure the redis server, instead of the gemfire
properties, which have now been removed.

3 months agoGEODE-10101: Update benchmark baseline
Dick Cavender [Fri, 18 Mar 2022 16:50:11 +0000 (09:50 -0700)] 
GEODE-10101: Update benchmark baseline

Updates the default benchmark baseline on develop to 1.14.4

3 months agoGEODE-10089: set version number to 1.15 to correspond with develop
Owen Nichols [Thu, 17 Mar 2022 00:08:23 +0000 (17:08 -0700)] 
GEODE-10089: set version number to 1.15 to correspond with develop

3 months agoGEODE-10080: Upgrade jedis to 4.1.1 (#165)
Eric Zoerner [Thu, 10 Mar 2022 17:44:21 +0000 (09:44 -0800)] 
GEODE-10080: Upgrade jedis to 4.1.1 (#165)

3 months agoFixes negative index. (#164)
Jacob Barrett [Sat, 5 Mar 2022 00:21:53 +0000 (16:21 -0800)] 
Fixes negative index. (#164)

4 months agoFix Jedis version to the version defined by benchmarks
Jens Deppe [Fri, 18 Feb 2022 18:18:32 +0000 (10:18 -0800)] 
Fix Jedis version to the version defined by benchmarks

- Develop has moved to Jedis 4.1.1 which requires code changes here.
  Since benchmark runs against the 'baseline' and develop both just use
  benchmarks at develop the benchmark code cannot be updated until the
  baseline is moved beyond the Jedis 4.1.1 change.

4 months agoGeode for Redis PubSub Benchmarks (#162)
Eric Zoerner [Fri, 4 Feb 2022 19:44:28 +0000 (11:44 -0800)] 
Geode for Redis PubSub Benchmarks (#162)

The new benchmarks are currently turned off due to high variation in average latency which would fail the CI threshold. We will revisit this in the future. For other uses these benchmarks can provide throughput data.

5 months agoupdate default benchmark baseline on develop
Dick Cavender [Wed, 26 Jan 2022 00:00:04 +0000 (16:00 -0800)] 
update default benchmark baseline on develop

6 months agoupdate default benchmark baseline on develop
Owen Nichols [Wed, 15 Dec 2021 23:50:26 +0000 (15:50 -0800)] 
update default benchmark baseline on develop

6 months agoupdate default benchmark baseline on develop
Owen Nichols [Sat, 11 Dec 2021 02:37:52 +0000 (18:37 -0800)] 
update default benchmark baseline on develop

6 months agoRevert "GEODE-9862: protect from jgit6.0.0 when building older Geodes for benchmarks"
Jens Deppe [Tue, 30 Nov 2021 23:08:37 +0000 (15:08 -0800)] 
Revert "GEODE-9862: protect from jgit6.0.0 when building older Geodes for benchmarks"

This reverts commit caa17e273b04eabf13b66a014042b73f97e457d0.

6 months agoGEODE-9862: protect from jgit6.0.0 when building older Geodes for benchmarks
Sean Goller [Tue, 30 Nov 2021 17:29:00 +0000 (09:29 -0800)] 
GEODE-9862: protect from jgit6.0.0 when building older Geodes for benchmarks

7 months agoGEODE-9291: Add benchmark for Zrem (#160)
Eric Zoerner [Fri, 5 Nov 2021 15:02:13 +0000 (08:02 -0700)] 
GEODE-9291: Add benchmark for Zrem (#160)

7 months agoGEODE-9796: workaround for cgroups issue
Owen Nichols [Thu, 4 Nov 2021 16:01:53 +0000 (09:01 -0700)] 
GEODE-9796: workaround for cgroups issue

7 months agoGEODE-9291 sorted set benchmarks - more benchmarks (#159)
Eric Zoerner [Fri, 29 Oct 2021 16:43:47 +0000 (09:43 -0700)] 
GEODE-9291 sorted set benchmarks - more benchmarks (#159)

 - Added benchmarks for ZRangeByScore and mixed ZAdd and ZRange operations

7 months agoGEODE-9291 Benchmarks for sorted sets (#158)
Eric Zoerner [Wed, 27 Oct 2021 16:46:23 +0000 (09:46 -0700)] 
GEODE-9291 Benchmarks for sorted sets (#158)

Basic ZADD and ZRANGE benchmarks for redis sorted sets.

8 months agoGEODE-9773: Update Radish region name
Jens Deppe [Tue, 26 Oct 2021 19:41:39 +0000 (12:41 -0700)] 
GEODE-9773: Update Radish region name

8 months agoGEODE-9175: Reducing the noise from the logging benchmark framework (#157)
Dan Smith [Thu, 14 Oct 2021 17:11:25 +0000 (10:11 -0700)] 
GEODE-9175: Reducing the noise from the logging benchmark framework (#157)

Changing a bunch of debug output to info level to reduce the amount of
noise the benchmark framework is printing out.

This is not the full scope of GEODE-9195, which also is asking to
print out progress as the test runs.

8 months agoSuppressing extra output in the dump_results.sh script
Dan Smith [Thu, 14 Oct 2021 04:13:47 +0000 (21:13 -0700)] 
Suppressing extra output in the dump_results.sh script

Suppressing gradle and pushd/popd logging in the dump_results.sh script.

8 months agoSlight tweak to the widths in DumpResults script
Dan Smith [Wed, 13 Oct 2021 22:12:29 +0000 (15:12 -0700)] 
Slight tweak to the widths in DumpResults script

Increasing the widthe of the test name column to accomodate our largest test
names. Eventually we could switch to a more intelligent table printing library
but this works for the moment.

8 months agomove the baseline version to the most recently released version, currently 1.14.0...
Hale Bales [Mon, 11 Oct 2021 23:18:26 +0000 (16:18 -0700)] 
move the baseline version to the most recently released version, currently 1.14.0 (#156)

8 months agoGEODE-9567: rename Redis compatibility
Owen Nichols [Sat, 9 Oct 2021 03:20:25 +0000 (20:20 -0700)] 
GEODE-9567: rename Redis compatibility

8 months agoGEODE-9567: rename Redis compatibility
Ray Ingles [Fri, 8 Oct 2021 16:03:05 +0000 (12:03 -0400)] 
GEODE-9567: rename Redis compatibility

8 months agoRemoving references to Geode internal classes (#155)
Dan Smith [Thu, 7 Oct 2021 18:06:21 +0000 (11:06 -0700)] 
Removing references to Geode internal classes (#155)

Geode internal classes shouldn't be used in the benchmarks. Using them makes it
harder to run tests against different versions of geode, since we don't make
garuantees about preserving the APIs of these classes between versions.

9 months agoMerge pull request #154 from jdeppe-pivotal/feature/reduce-file-copy-logging
Jens Deppe [Wed, 1 Sep 2021 22:50:49 +0000 (15:50 -0700)] 
Merge pull request #154 from jdeppe-pivotal/feature/reduce-file-copy-logging

Remove file copy logging

10 months agoRemove file copy logging 154/head
Jens Deppe [Thu, 19 Aug 2021 20:08:26 +0000 (13:08 -0700)] 
Remove file copy logging

- This makes up almost ALL of the run output which doesn't seem to serve
  any useful purpose.

12 months agoAdd key range support to more benchmarks.
Jacob Barrett [Tue, 8 Jun 2021 18:00:33 +0000 (11:00 -0700)] 
Add key range support to more benchmarks.

13 months agoRename Redis region.
Jacob Barrett [Tue, 18 May 2021 23:39:21 +0000 (16:39 -0700)] 
Rename Redis region.

13 months agoAdd ManualRedisTopology test for hosts and ports (#151)
Sarah [Mon, 17 May 2021 16:27:42 +0000 (12:27 -0400)] 
Add ManualRedisTopology test for hosts and ports (#151)

13 months agoAdds Redis benchmarks. (#150)
Jacob Barrett [Mon, 17 May 2021 14:33:23 +0000 (07:33 -0700)] 
Adds Redis benchmarks. (#150)

* Adds framework for Redis based benchmarks.
* Supports benchmarking Redis, Geode and external Redis like systems.
* Supports Jedis and Lettuce clients.
* Tunable cluster sizing.

Co-authored-by: Dan Smith <dasmith@vmware.com>
13 months agoGEODE-8321: part 3 - remove adoptopenjdk
Owen Nichols [Mon, 10 May 2021 23:19:06 +0000 (16:19 -0700)] 
GEODE-8321: part 3 - remove adoptopenjdk

14 months agoGEODE-9176: Pass TEST_HOSTS with benchmark. prefix
Dan Smith [Mon, 26 Apr 2021 22:12:45 +0000 (15:12 -0700)] 
GEODE-9176: Pass TEST_HOSTS with benchmark. prefix

It looks like this property was getting passed through with the wrong prefix.
Setting the system property to be benchmark.TEST_HOSTS

14 months agoGEODE-9176: Automatically pass properties to benchmark JVMs (#149)
Dan Smith [Mon, 26 Apr 2021 18:23:37 +0000 (11:23 -0700)] 
GEODE-9176: Automatically pass properties to benchmark JVMs (#149)

Converting all of the system properties we use in the benchmarks to start with the benchmark
prefix. Changing the gradle build to copy all benchmark.* properties as system
properties in the tst.

Getting rid of problematic org.json dependency, and also making
sure we capture *all* system properties that might effect the behavior of the
test.

Adding a way to automatically set system properties in test JVMs. Just add a property
with the prefix benchmark.system.ROLE, where ROLE is the role of jvms to target. Eg

benchmark.system.server.gemfire.disablePartitionedRegionBucketAck=true would set
gemfire.disablePartitionedRegionBucketAck=true  in the server JVMs.

Logging all benchmark properties during test run

14 months agoGEODE-9177: Adding a new analyzer that dumps data in a tabular format (#148)
Dan Smith [Wed, 21 Apr 2021 18:36:09 +0000 (11:36 -0700)] 
GEODE-9177: Adding a new analyzer that dumps data in a tabular format (#148)

This analyzer reads all of the benchmark results and prints one row for each
result. It does not do a comparison with a baseline.  For example:

dump_results.sh resultDir1 resultDir2 resultDir3
> Task :harness:dumpRun
Directory                                Benchmark                      ops/sec          avg latency(ms)  99%% latency(ms)
partition-10-1-thread-1K-values          PartitionBenchmark             52143.62         0.0763           0.1348
partition-10-16-thread-1K-value          PartitionBenchmark             232304.35        0.2747           0.6400
partition-10-4-thread-1K-values          PartitionBenchmark             134713.40        0.1182           0.2440

15 months agoDisable P2pPartitionedPutLongBenchmark (#140)
Donal Evans [Thu, 25 Mar 2021 18:07:44 +0000 (11:07 -0700)] 
Disable P2pPartitionedPutLongBenchmark (#140)

- This benchmark is unstable and does not produce reliable results. See
GEODE-8950 for further details.

Authored-by: Donal Evans <doevans@vmware.com>
15 months agoPrevent Travis build timeout
Donal Evans [Thu, 4 Mar 2021 02:24:49 +0000 (18:24 -0800)] 
Prevent Travis build timeout

- Add output after each test completes, as Travis declares a task hung
if no output is written for 10 minutes, which is longer than the build
and tests take to complete

15 months agoPrevent Travis build timeout 139/head 142/head 143/head
Donal Evans [Wed, 3 Mar 2021 20:16:48 +0000 (12:16 -0800)] 
Prevent Travis build timeout

- Add output after each test completes, as Travis declares a task hung
if no output is written for 10 minutes, which is longer than the build
and tests take to complete

Authored-by: Donal Evans <doevans@vmware.com>
16 months agoBumping copyright year to 2021
Owen Nichols [Fri, 12 Feb 2021 00:27:17 +0000 (16:27 -0800)] 
Bumping copyright year to 2021

19 months agorevert change to output file naming (#138)
Hale Bales [Thu, 19 Nov 2020 06:42:44 +0000 (22:42 -0800)] 
revert change to output file naming (#138)

19 months agoAdds a router to SNI proxy topology. (#137)
Jacob Barrett [Tue, 17 Nov 2020 22:06:32 +0000 (14:06 -0800)] 
Adds a router to SNI proxy topology. (#137)

* Update JUnit
* Adds withRouter option.
* Adds withRouterImage option.

19 months agoImproves on SNI proxy benchmarking. (#136)
Jacob Barrett [Tue, 17 Nov 2020 19:24:36 +0000 (11:24 -0800)] 
Improves on SNI proxy benchmarking. (#136)

* Adds support for Envoy as proxy server.
* Adds support for manually configured and managed proxy.
* Adds ability to switch between proxy implementations.
* Adds support for overriding docker image:tag for proxy implementations.

19 months agoMoved #cloud-config to be first line so EC2 knows it is a cloud file
Robert Houghton [Mon, 16 Nov 2020 17:57:42 +0000 (09:57 -0800)] 
Moved #cloud-config to be first line so EC2 knows it is a cloud file

License is secondary. Sorry RAT

Co-authored-by: Robert Houghton <rhoughton@pivotal.io>
Co-authored-by: Sean Goller <sgoller@pivotal.io>
19 months agoAdds peer to peer benchmarks. (#135)
Jacob Barrett [Thu, 12 Nov 2020 15:35:40 +0000 (07:35 -0800)] 
Adds peer to peer benchmarks. (#135)

* Adds P2P put test benchmark.
* Adds P2P get test benchmark.
* Adds put bytes benchmark.
* Adds SSL parameters to benchmarks.
* Fixes API access for old releases.

22 months agoRevert "allow AMI picking with arbitrary username accounts (#132)" (#133) feature/GEODE-7665
Robert Houghton [Wed, 12 Aug 2020 17:47:56 +0000 (10:47 -0700)] 
Revert "allow AMI picking with arbitrary username accounts (#132)" (#133)

This reverts commit 3e3bdd3a371e400db0ce2ce2d736ab06f9244ed1.

22 months agoallow AMI picking with arbitrary username accounts (#132)
Robert Houghton [Wed, 12 Aug 2020 16:58:16 +0000 (09:58 -0700)] 
allow AMI picking with arbitrary username accounts (#132)

* allow AMI picking with arbitrary username accounts
* improved run directory settings

23 months agoGEODE-8348: Add support for customizable purpose tag. (#131)
Sean Goller [Wed, 15 Jul 2020 22:35:06 +0000 (15:35 -0700)] 
GEODE-8348: Add support for customizable purpose tag. (#131)

* GEODE-8348: Add support for customizable purpose tag.

* GEODE-8348: purpose value fixes.

23 months agoRevert "Enable long bench"
Jacob Barrett [Wed, 15 Jul 2020 06:22:56 +0000 (23:22 -0700)] 
Revert "Enable long bench"

This reverts commit 4aee6e218dd16c2d41f0f9e0b009236b349e2aae.

23 months agoEnable long bench
Jacob Barrett [Wed, 15 Jul 2020 06:20:12 +0000 (23:20 -0700)] 
Enable long bench

2 years agoAdd liberica JDKs to image.
Sean Goller [Fri, 26 Jun 2020 21:05:12 +0000 (14:05 -0700)] 
Add liberica JDKs to image.

Authored-by: Sean Goller <sgoller@pivotal.io>
2 years agoTLS SNI support via "withSniProxy" Option (#129)
Bill Burcham [Mon, 8 Jun 2020 21:14:41 +0000 (14:14 -0700)] 
TLS SNI support via "withSniProxy" Option (#129)

2 years agoBumping copyright year to 2020 rel/v1.13.0 rel/v1.13.0.RC1 rel/v1.13.1 rel/v1.13.1.RC1 rel/v1.13.1.RC2
David Barnes [Mon, 4 May 2020 17:01:00 +0000 (10:01 -0700)] 
Bumping copyright year to 2020

2 years agoGEODE-8006 Add .asf.yaml to control notifications
Anthony Baker [Wed, 22 Apr 2020 15:29:02 +0000 (08:29 -0700)] 
GEODE-8006 Add .asf.yaml to control notifications

2 years agoEnabling logging for the geode-benchmarks (#128)
Dan Smith [Tue, 21 Apr 2020 16:27:36 +0000 (09:27 -0700)] 
Enabling logging for the geode-benchmarks (#128)

The benchmarks we not logging any of the geode logs, they reported "ERROR
StatusLogger No Log4j 2 configuration file found."

I believe this was broken when we switched to having geode-log4j as a separate
module. Adding that module in as a dependency, which reenables standard geode
logging.

2 years agoGEODE-7716: Use gradlew that does not require a checked in jar dunitrunner-readme support/1.10 rel/v1.12.0 rel/v1.12.0.RC1 rel/v1.12.0.RC2 rel/v1.12.0.RC3 rel/v1.12.0.RC4
Dan Smith [Fri, 17 Jan 2020 18:29:50 +0000 (10:29 -0800)] 
GEODE-7716: Use gradlew that does not require a checked in jar

When publishing a source distribution, we should not include any jars in the
distribution. For other geode projects, (eg geode-examples) we use this gradlew
which downloads the gradle-wrapper.jar file was well.

2 years agoupgrade gradle from 5.0 to 5.4 (#126)
Owen Nichols [Wed, 22 Jan 2020 17:28:03 +0000 (09:28 -0800)] 
upgrade gradle from 5.0 to 5.4 (#126)

* update gradle from 5.0 to 5.4 to match geode and geode-examples

2 years agofix output dir motification (#124)
Helena Bales [Mon, 23 Dec 2019 19:45:00 +0000 (11:45 -0800)] 
fix output dir motification (#124)

modify output dir to be unique for running in CI

2 years agoinstall JDK11 on image and fix image script issue with dpkg (#123)
Helena Bales [Wed, 18 Dec 2019 21:49:20 +0000 (13:49 -0800)] 
install JDK11 on image and fix image script issue with dpkg (#123)

Signed-off-by: Sean Goller <sgoller@pivotal.io>
2 years agoFixes exclude of LongBenchmarks
Jacob Barrett [Tue, 17 Dec 2019 18:19:17 +0000 (10:19 -0800)] 
Fixes exclude of LongBenchmarks

2 years agoexclude flaky LongBenchmarks from running in CI or locally (#122)
Helena Bales [Fri, 13 Dec 2019 17:57:50 +0000 (09:57 -0800)] 
exclude flaky LongBenchmarks from running in CI or locally (#122)

2 years agoExclude failing NoopBenchmark (#121)
Helena Bales [Thu, 12 Dec 2019 20:59:05 +0000 (12:59 -0800)] 
Exclude failing NoopBenchmark (#121)

NoopBenchmark has some fundamental issues that make it fail constantly.
In recent testing, it passed significantly more often that it has been
in Apache Geode CI, so it needs to be excluded until it can be fixed.

2 years agoGEODE-7554: Add retry mechanism for failed tests (#120)
Helena Bales [Wed, 11 Dec 2019 22:13:41 +0000 (14:13 -0800)] 
GEODE-7554: Add retry mechanism for failed tests (#120)

* write failed tests to a file
* enable all benchmarks
* update help message for --ci option in run_against_baseline.sh

2 years agoConfirm and retry AWS instance allocations. (#119)
Helena Bales [Fri, 6 Dec 2019 23:14:44 +0000 (15:14 -0800)] 
Confirm and retry AWS instance allocations. (#119)

Confirm and retry AWS instance allocations.

Retry with timeouts, confirming the expected number of instances have
been allocated.

Retry with timeouts, confirming the expected number of instances are in
the running state.

Signed-off-by: Robert Houghton <rhoughton@pivotal.io>
2 years agostabilize putall tests (#118)
Helena Bales [Mon, 2 Dec 2019 19:35:04 +0000 (11:35 -0800)] 
stabilize putall tests (#118)

2 years agorewrite createSecurityGroup method in LaunchCluster to fix logic errors (#117)
Helena Bales [Mon, 25 Nov 2019 18:52:17 +0000 (10:52 -0800)] 
rewrite createSecurityGroup method in LaunchCluster to fix logic errors (#117)

2 years agoretry creating the security group (#116)
Helena Bales [Fri, 15 Nov 2019 19:04:50 +0000 (11:04 -0800)] 
retry creating the security group (#116)

retry creating the security group

2 years agotemporarily exclude flaky tests
Helena A. Bales [Fri, 25 Oct 2019 18:36:24 +0000 (11:36 -0700)] 
temporarily exclude flaky tests

2 years agoexclude broken Noop test until it can be fixed
Helena A. Bales [Thu, 24 Oct 2019 16:45:07 +0000 (09:45 -0700)] 
exclude broken Noop test until it can be fixed

2 years agoUses full dedicated host and adds more configuration options. (#114)
Jacob Barrett [Tue, 22 Oct 2019 16:43:03 +0000 (09:43 -0700)] 
Uses full dedicated host and adds more configuration options. (#114)

* Use full c5-18-xlarge instance type on dedicated host.
* Adds -PwithThread to override benchmark threads.
* Adds -PwithHeap to override max heap.
* Adds -PwithWarmup to override warmup time.
* Adds -PwithDuration to override test duration.

2 years agoFixes target resolution.
Jacob Barrett [Wed, 16 Oct 2019 20:45:43 +0000 (13:45 -0700)] 
Fixes target resolution.

2 years agoAllow repository to by rsync'd from local host rather than cloned.
Jacob Barrett [Tue, 15 Oct 2019 22:57:50 +0000 (15:57 -0700)] 
Allow repository to by rsync'd from local host rather than cloned.

2 years agoAdds more command line configuration options. (#112)
Jacob Barrett [Mon, 30 Sep 2019 16:58:18 +0000 (09:58 -0700)] 
Adds more command line configuration options. (#112)

* Support for selection GC from CMS, G1, Shenandoah, or Z.
* Support for specifying JMV heap size.

* Refactors JVM argument configuration.
* Improved new GC logging

2 years agoGet first host by array reference (#109)
Jacob Barrett [Thu, 19 Sep 2019 19:14:46 +0000 (12:14 -0700)] 
Get first host by array reference (#109)

2 years agoGet first host by array reference (#109)
Jacob Barrett [Thu, 19 Sep 2019 19:02:50 +0000 (12:02 -0700)] 
Get first host by array reference (#109)

2 years agoCreate dedicated hosts (#109)
Jacob Barrett [Wed, 18 Sep 2019 02:52:50 +0000 (19:52 -0700)] 
Create dedicated hosts (#109)

2 years agoRevert "Create dedicated hosts (#109)"
Jacob Barrett [Wed, 18 Sep 2019 03:16:03 +0000 (20:16 -0700)] 
Revert "Create dedicated hosts (#109)"

This reverts commit 410f23a16886a3fb33fd5eb5cedc9620e092dcba.

2 years agoCreate dedicated hosts (#109)
Jacob Barrett [Wed, 18 Sep 2019 02:52:50 +0000 (19:52 -0700)] 
Create dedicated hosts (#109)

2 years agoIgnore PartitionedFunctionExecutionBenchmark
Jacob Barrett [Tue, 17 Sep 2019 21:56:51 +0000 (14:56 -0700)] 
Ignore PartitionedFunctionExecutionBenchmark

2 years agoRevert "Create dedicated hosts (#109)"
Jacob Barrett [Mon, 16 Sep 2019 18:39:38 +0000 (11:39 -0700)] 
Revert "Create dedicated hosts (#109)"

This reverts commit 2adaf56de71fd4bdceec97c2387eaacb4639e63c.

2 years agoCreate dedicated hosts (#109)
Kamilla Aslami [Fri, 13 Sep 2019 20:33:41 +0000 (13:33 -0700)] 
Create dedicated hosts (#109)

Co-authored-by: Kamilla Aslami <kaslami@pivotal.io>
Co-authored-by: Jacob Barrett <jbarrett@pivotal.io>
2 years agoRevert "Prevent too many keys being offered to ssh server (#103)"
Helena A. Bales [Fri, 13 Sep 2019 16:47:12 +0000 (09:47 -0700)] 
Revert "Prevent too many keys being offered to ssh server (#103)"

This reverts commit 5062dccfa91cb6acebb18cd396810932bce1163a.

This commit appears to cause benchmark task in CI to timeout. Reverting
while we investigate root cause.

2 years agoPrevent too many keys being offered to ssh server (#103)
Donal Evans [Thu, 12 Sep 2019 18:01:03 +0000 (11:01 -0700)] 
Prevent too many keys being offered to ssh server (#103)

2 years agoCheck that security group exists (#107)
Kamilla Aslami [Thu, 12 Sep 2019 17:57:44 +0000 (10:57 -0700)] 
Check that security group exists (#107)

* Check that security group exists
* Throw an exception after max retries + refactoring

2 years agoAdd *LongBenchmark (#110)
Kamilla Aslami [Thu, 12 Sep 2019 17:52:13 +0000 (10:52 -0700)] 
Add *LongBenchmark (#110)

Add prefill and benchmarks using Longs

2 years agoEnabled PutAll benchmarks (#106)
Kamilla Aslami [Fri, 6 Sep 2019 17:31:22 +0000 (10:31 -0700)] 
Enabled PutAll benchmarks (#106)

2 years agoStabilizes query benchmarks (#104)
Kamilla Aslami [Thu, 5 Sep 2019 00:30:24 +0000 (17:30 -0700)] 
Stabilizes query benchmarks (#104)

* Pre-fill region from client so that all servers have PDX instances
* Change query range and number of threads for query benchmarks

2 years agoFix hash map keys in PutAllTask (#105)
Kamilla Aslami [Tue, 3 Sep 2019 22:49:04 +0000 (15:49 -0700)] 
Fix hash map keys in PutAllTask (#105)

2 years agoStabilizes function benchmarks (#101)
Jacob Barrett [Tue, 27 Aug 2019 23:29:42 +0000 (16:29 -0700)] 
Stabilizes function benchmarks (#101)

* New LongRange object for managing ranges of longs.
* Only get 1 value.
* Balanced CPU utilization and threads.

2 years agoExpanding and clarifying readme files for using aws and scripts. (#102)
Donal Evans [Mon, 26 Aug 2019 19:11:46 +0000 (12:11 -0700)] 
Expanding and clarifying readme files for using aws and scripts. (#102)

2 years ago Fixed spotless configuration to support groovy and gradle. (#100)
Kamilla Aslami [Wed, 21 Aug 2019 20:06:25 +0000 (13:06 -0700)] 
 Fixed spotless configuration to support groovy and gradle. (#100)

* Fixed spotless configuration to support groovy and gradle
* Spotless applied

Co-authored-by: Murtuza Boxwala <mboxwala@pivotal.io>
Co-authored-by: Kamilla Aslami <kaslami@pivotal.io>
Co-authored-by: Jacob Barrett <jbarrett@pivotal.io>
2 years agoRemove unattended upgrades so ubuntu doesn't try to upgrade itself mid
Sean Goller [Mon, 19 Aug 2019 19:21:18 +0000 (12:21 -0700)] 
Remove unattended upgrades so ubuntu doesn't try to upgrade itself mid
job.

Authored-by: Sean Goller <sgoller@pivotal.io>
2 years agoLower thread count for improved stability. (#98)
Jacob Barrett [Mon, 12 Aug 2019 19:44:48 +0000 (12:44 -0700)] 
Lower thread count for improved stability. (#98)

By reducing thread count to 10x CPUs we found that on the same instances we only lose about 4% of throughput but seem more than 40% improvement in latency. More importantly the variation between runs drops significantly. This variation was greater than 10% on Java 11 and 3% on Java 8. When lowered to 10x threads we see closer to 1% variation for both Java 11 and 8.

2 years agodisable replicated and partitioned putAll benchmark due to flakiness (#95)
Helena Bales [Thu, 25 Jul 2019 17:43:46 +0000 (10:43 -0700)] 
disable replicated and partitioned putAll benchmark due to flakiness (#95)

2 years agoAdded --baselineDir and branchDir options to analyze_tests script (#93)
Kamilla Aslami [Thu, 25 Jul 2019 16:35:39 +0000 (09:35 -0700)] 
Added --baselineDir and branchDir options to analyze_tests script (#93)

2 years agoUser can run benchmarks with SecurityManager (#91)
Murtuza Boxwala [Mon, 22 Jul 2019 19:31:28 +0000 (15:31 -0400)] 
User can run benchmarks with SecurityManager (#91)

Co-authored-by: Murtuza Boxwala <mboxwala@pivotal.io>
Co-authored-by: Kamilla Aslami <kaslami@pivotal.io>
2 years agoGenerate a self-signed certificate for SSL (#90)
Murtuza Boxwala [Mon, 22 Jul 2019 16:27:29 +0000 (12:27 -0400)] 
Generate a self-signed certificate for SSL (#90)

Co-authored-by: Murtuza Boxwala <mboxwala@pivotal.io>
Co-authored-by: Kamilla Aslami <kaslami@pivotal.io>
2 years agoUser can run benchmarks with alternative JVM (#89)
Kamilla Aslami [Wed, 17 Jul 2019 16:37:13 +0000 (09:37 -0700)] 
User can run benchmarks with alternative JVM (#89)

* Use alternative JVM for benchmark/test
* Use java.home for executing remote jvm.
* Disable GC logging if not Java 8
* Updated the README.md to be clearer.
* Added ClientServerTopologyTest file

Co-authored-by: Murtuza Boxwala <mboxwala@pivotal.io>
Co-authored-by: Kamilla Aslami <kaslami@pivotal.io>