geode-benchmarks.git
3 years agoAdd expire_clusters script.
Helena Bales [Fri, 10 May 2019 18:56:39 +0000 (11:56 -0700)] 
Add expire_clusters script.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoMerge pull request #80 from balesh2/fixREADME
Helena Bales [Mon, 29 Apr 2019 18:38:14 +0000 (11:38 -0700)] 
Merge pull request #80 from balesh2/fixREADME

fix readme for copy to cluster script

3 years agofix readme for copy to cluster script 80/head
Helena A. Bales [Mon, 29 Apr 2019 18:26:01 +0000 (11:26 -0700)] 
fix readme for copy to cluster script

3 years agoDisable replicated query and function benchmarks.
Jacob Barrett [Fri, 26 Apr 2019 03:02:19 +0000 (20:02 -0700)] 
Disable replicated query and function benchmarks.

Re-enable partitioned.

3 years agoDisable partitioned query and function benchmarks.
Jacob Barrett [Fri, 26 Apr 2019 03:00:46 +0000 (20:00 -0700)] 
Disable partitioned query and function benchmarks.

3 years agoDisable query again
Jacob Barrett [Wed, 24 Apr 2019 21:43:18 +0000 (14:43 -0700)] 
Disable query again

3 years agoSpotless
Jacob Barrett [Wed, 24 Apr 2019 19:11:02 +0000 (12:11 -0700)] 
Spotless

3 years agoMerge pull request #78 from moleske/moleske-patch-1
Helena Bales [Wed, 24 Apr 2019 18:55:29 +0000 (11:55 -0700)] 
Merge pull request #78 from moleske/moleske-patch-1

Remove dead GCP link

3 years agoReduce threads on query benchmarks.
Jacob Barrett [Wed, 24 Apr 2019 16:18:42 +0000 (09:18 -0700)] 
Reduce threads on query benchmarks.

3 years agoCleanup PDX objects.
Jacob Barrett [Wed, 24 Apr 2019 16:02:53 +0000 (09:02 -0700)] 
Cleanup PDX objects.

Reduces wasted allocations on hashCode().
Reduces wast in allocations on toString().

3 years agoRe-enable query benchmarks.
Jacob Barrett [Wed, 24 Apr 2019 05:04:17 +0000 (22:04 -0700)] 
Re-enable query benchmarks.

3 years agoRemove dead GCP link 78/head
M. Oleske [Tue, 23 Apr 2019 22:33:32 +0000 (15:33 -0700)] 
Remove dead GCP link

3 years agoMove threads configuration.
Jacob Barrett [Fri, 19 Apr 2019 14:03:11 +0000 (07:03 -0700)] 
Move threads configuration.
Run fewer threads on function and queries.
Refactor common elements.

3 years agoSpotless
Jacob Barrett [Fri, 19 Apr 2019 18:12:17 +0000 (11:12 -0700)] 
Spotless

3 years agoDisables function benchmarks.
Jacob Barrett [Fri, 19 Apr 2019 18:03:24 +0000 (11:03 -0700)] 
Disables function benchmarks.

3 years agoUse larger disk
Jacob Barrett [Fri, 19 Apr 2019 02:48:26 +0000 (19:48 -0700)] 
Use larger disk

3 years agoAlways reset geode repo.
Jacob Barrett [Fri, 19 Apr 2019 01:03:34 +0000 (18:03 -0700)] 
Always reset geode repo.

3 years agoDisable *IndexedQueryBenchmark
Jacob Barrett [Thu, 18 Apr 2019 23:39:28 +0000 (16:39 -0700)] 
Disable *IndexedQueryBenchmark

Currently unstable.

3 years agoDisable *NonIndexedQueryBenchmark
Jacob Barrett [Thu, 18 Apr 2019 23:37:06 +0000 (16:37 -0700)] 
Disable *NonIndexedQueryBenchmark

Currently unstable.

3 years agoAlways cleanup benchmark directory.
Jacob Barrett [Thu, 18 Apr 2019 22:58:22 +0000 (15:58 -0700)] 
Always cleanup benchmark directory.

3 years agoFixes install of geode artifacts.
Jacob Barrett [Thu, 18 Apr 2019 21:38:00 +0000 (14:38 -0700)] 
Fixes install of geode artifacts.

Uses publishToMavenLocal when available or install on older builds.

Updates existing repo rather than deleting it.

3 years agoFixes parameter parsing in AWS scripts.
Jacob Barrett [Thu, 18 Apr 2019 16:46:37 +0000 (09:46 -0700)] 
Fixes parameter parsing in AWS scripts.

3 years agoGEODE-6628: Increase thread count (#76)
Helena Bales [Fri, 12 Apr 2019 19:58:11 +0000 (12:58 -0700)] 
GEODE-6628: Increase thread count (#76)

3 years agoRAT fixes. (#75)
Sean Goller [Wed, 10 Apr 2019 16:37:30 +0000 (09:37 -0700)] 
RAT fixes. (#75)

Co-authored-by: Helena Bales <hbales@pivotal.io>
Co-authored-by: Sean Goller <sgoller@pivotal.io>
3 years agofind and destroy clusters that were started more than 7 days ago (#74)
Sean Goller [Tue, 9 Apr 2019 23:45:17 +0000 (16:45 -0700)] 
find and destroy clusters that were started more than 7 days ago (#74)

Co-authored-by: Helena Bales <hbales@pivotal.io>
Co-authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoFinish up unique identification of builds. (#73)
Sean Goller [Tue, 9 Apr 2019 21:49:49 +0000 (14:49 -0700)] 
Finish up unique identification of builds. (#73)

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoMetadata generation tweaks and database submission code (#72)
Sean Goller [Fri, 22 Mar 2019 19:31:11 +0000 (12:31 -0700)] 
Metadata generation tweaks and database submission code (#72)

* Refactor and tweaking database submission.
Co-authored-by: Sean Goller <sgoller@pivotal.io>
Co-authored-by: Helena Bales <hbales@pivotal.io>
3 years agoMerge pull request #71 from balesh2/updateREADME
Helena Bales [Wed, 20 Mar 2019 22:48:45 +0000 (15:48 -0700)] 
Merge pull request #71 from balesh2/updateREADME

update profiling instructions in the readme

3 years agoadd info on path to profiler 71/head
Helena A. Bales [Wed, 20 Mar 2019 22:31:21 +0000 (15:31 -0700)] 
add info on path to profiler

3 years agoupdate profiling instructions in the readme
Helena A. Bales [Wed, 20 Mar 2019 20:45:54 +0000 (13:45 -0700)] 
update profiling instructions in the readme

* add initial configuration information for YourKit
* fix path to private key when setting up connection authentication in
YourKit

3 years agoMerge pull request #70 from balesh2/updateREADME
Helena Bales [Thu, 14 Mar 2019 17:53:43 +0000 (10:53 -0700)] 
Merge pull request #70 from balesh2/updateREADME

update the readme with instructions for profiling

3 years agoupdate the readme with instructions for profiling 70/head
Helena A. Bales [Tue, 12 Mar 2019 22:07:18 +0000 (15:07 -0700)] 
update the readme with instructions for profiling

3 years agoAdd build_identifier.
Sean Goller [Tue, 12 Mar 2019 18:35:07 +0000 (11:35 -0700)] 
Add build_identifier.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoGEODE-6458: add profiler argument support (#65)
Helena Bales [Tue, 12 Mar 2019 17:18:53 +0000 (10:18 -0700)] 
GEODE-6458: add profiler argument support (#65)

* Added profiler argument support.

* take profile snapshot on jvm close and copy snapshot with benchmark results

Signed-off-by: Jacob Barrett <jbarrett@pivotal.io>
* More string replacements.

* remove dependency on StringUtil since Geode may not have that

Signed-off-by: Helena A. Bales <hbales@pivotal.io>
* GEODE-6458: add options for profiler

add options to run with a profiler, attached to all members.

Signed-off-by: Sean Goller <sgoller@pivotal.io>
3 years agoMerge pull request #68 from apache/feature/GEODE-6508
Darrel Schneider [Tue, 12 Mar 2019 00:12:58 +0000 (17:12 -0700)] 
Merge pull request #68 from apache/feature/GEODE-6508

GEODE-6508: update aws README.md

3 years agoMerge pull request #69 from balesh2/GEODE-6510
Helena Bales [Mon, 11 Mar 2019 23:22:16 +0000 (16:22 -0700)] 
Merge pull request #69 from balesh2/GEODE-6510

GEODE-6510: update scripts to use new key location

3 years agoGEODE-6510: update scripts to use new key location 69/head
Helena A. Bales [Mon, 11 Mar 2019 21:35:00 +0000 (14:35 -0700)] 
GEODE-6510: update scripts to use new key location

3 years agoGEODE-6508: update aws README.md 68/head
Darrel Schneider [Mon, 11 Mar 2019 21:11:37 +0000 (14:11 -0700)] 
GEODE-6508: update aws README.md

3 years agoGEODE-6502: update README (#67)
Helena Bales [Mon, 11 Mar 2019 16:43:34 +0000 (09:43 -0700)] 
GEODE-6502: update README (#67)

Update the readme to reflect all recent script changes, including
changing from positional to named arguments for launching and destroying
clusters, the addition of a --ci option for launching and destroying
clusters for use in continuous integration, and the addition of long
command line options for all scripts.

3 years agoGEODE-6480: break so that options are passed on (#66)
Helena Bales [Mon, 11 Mar 2019 16:43:16 +0000 (09:43 -0700)] 
GEODE-6480: break so that options are passed on (#66)

break out of the case statement so that options after '--' when running
tests and running against the baseline are passed on to gradle.

3 years agoGEODE-6502: update README 67/head
Helena A. Bales [Fri, 8 Mar 2019 20:53:38 +0000 (12:53 -0800)] 
GEODE-6502: update README

Update the readme to reflect all recent script changes, including
changing from positional to named arguments for launching and destroying
clusters, the addition of a --ci option for launching and destroying
clusters for use in continuous integration, and the addition of long
command line options for all scripts.

3 years agoGEODE-6479: add username to aws keypair prefix (#62)
Helena Bales [Fri, 8 Mar 2019 18:35:44 +0000 (10:35 -0800)] 
GEODE-6479: add username to aws keypair prefix (#62)

* GEODE-6479: add username to aws keypair prefix

Get the current username from system properties and add it to the aws
prefix to make the instance owners more immediately identifiable. This
should help reduce the number of instances that are left running with no
clear owner.

If it is run in CI, the username does not get added.

Signed-off-by: Sean Goller <sgoller@pivotal.io>
3 years agoAdd python script to submit database results. (#64)
Sean Goller [Thu, 7 Mar 2019 17:28:47 +0000 (09:28 -0800)] 
Add python script to submit database results. (#64)

* Initial script to submit benchmark to database.

* Creates benchmark_build entry, but doesn't submit results yet.

Co-authored-by: Sean Goller <sgoller@pivotal.io>
Co-authored-by: Nabarun Nag <nabarunnag@pivotal.io>
* Add latency results.

Co-authored-by: Sean Goller <sgoller@pivotal.io>
Co-authored-by: Nabarun Nag <nabarunnag@pivotal.io>
* Add throughput, remove debug print()s.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoGEODE-6480: Just Do It Day: Use long cli options (#63)
Helena Bales [Wed, 6 Mar 2019 23:11:32 +0000 (15:11 -0800)] 
GEODE-6480: Just Do It Day: Use long cli options (#63)

Add long CLI options for scripts to run benchmarks on AWS instances.
Backwards compatible with previous short CLI options.

Signed-off-by: Robert Houghton <rhoughton@pivotal.io>
3 years agoChange bash tests to double brackets and fix baseline branch test. (#61)
Sean Goller [Fri, 1 Mar 2019 19:32:34 +0000 (11:32 -0800)] 
Change bash tests to double brackets and fix baseline branch test. (#61)

Co-authored-by: Sean Goller <sgoller@pivotal.io>
Co-authored-by: Nabarun Nag <nabarunnag@pivotal.io>
3 years agoClean up metadata generation. Add all test names to metadata. (#60)
Sean Goller [Fri, 1 Mar 2019 00:44:11 +0000 (16:44 -0800)] 
Clean up metadata generation. Add all test names to metadata. (#60)

Co-authored-by: Sean Goller <sgoller@pivotal.io>
Co-authored-by: Nabarun Nag <nabarunnag@pivotal.io>
3 years agoAdds copy to cluster script
Jacob Barrett [Thu, 28 Feb 2019 17:06:17 +0000 (09:06 -0800)] 
Adds copy to cluster script

3 years agoFix metadata (#59)
Sean Goller [Wed, 27 Feb 2019 23:28:09 +0000 (15:28 -0800)] 
Fix metadata (#59)

* Fix ssh options for running tests.

* Actually fix ssh options.

3 years agoFix ssh options for running tests. (#58)
Sean Goller [Wed, 27 Feb 2019 23:14:58 +0000 (15:14 -0800)] 
Fix ssh options for running tests. (#58)

3 years agoRefactor metadata production and management. (#57)
Sean Goller [Wed, 27 Feb 2019 22:56:45 +0000 (14:56 -0800)] 
Refactor metadata production and management. (#57)

* Tests now output metadata if the metadata file doesn't exist.
* benchmark run metadata.json is actually json.
* Create configuration directory to store cluster information.
* Move SSH private key for cluster to configuration directory.
* Create metadata for cluster, currently including an instance ID and
  the list of IPs in the cluster.
* Delete metadata when the cluster is destroyed.

3 years agoGEODE-6411: Fixes baseline version option.
Jacob Barrett [Wed, 27 Feb 2019 05:04:44 +0000 (21:04 -0800)] 
GEODE-6411: Fixes baseline version option.

3 years agoGEODE-6444: Store GC logs in files (#56)
Nabarun Nag [Mon, 25 Feb 2019 15:58:13 +0000 (07:58 -0800)] 
GEODE-6444: Store GC logs in files (#56)

* Store the GC logs in a file.
* Print the GC logs in the System.out / System.err in parallel while the benchmark is being executed in parallel.
* Using a thread pool for execution of runAsync and not depend on fork join pools
        * Disabling the dstat probe

Co-authored-by: Dan Smith <dsmith@pivotal.io>
Co-authored-by: nabarunnag <nag@cs.wisc.edu>
Co-authored-by: Jacob Barrett <jbarrett@pivotal.io>
3 years agoGEODE-6411: Fixes version/branch logic.
Jacob Barrett [Thu, 21 Feb 2019 00:07:40 +0000 (16:07 -0800)] 
GEODE-6411: Fixes version/branch logic.

3 years agoGEODE-6411: Fixes help
Jacob Barrett [Wed, 20 Feb 2019 20:13:59 +0000 (12:13 -0800)] 
GEODE-6411: Fixes help

- Adds run_on_cluster script
  `./run_on_cluster.sh -t cluster -- pkill java`

3 years agoGEODE-6411: Fixes quoting
Jacob Barrett [Wed, 20 Feb 2019 16:20:19 +0000 (08:20 -0800)] 
GEODE-6411: Fixes quoting

3 years agoGEODE-6411: Allow alternative repo (#55)
Jacob Barrett [Wed, 20 Feb 2019 04:54:50 +0000 (20:54 -0800)] 
GEODE-6411: Allow alternative repo (#55)

- Arguments after -- will be passed to benchmark gradle task.
  Example: Add `--tests MyTest` to run specific benchmark.

Authored-by: Jacob Barrett <jbarrett@pivotal.io>
3 years agoGEODE-6394: set pool idle-timeout to be infinite (#54)
Helena Bales [Fri, 15 Feb 2019 00:21:10 +0000 (16:21 -0800)] 
GEODE-6394: set pool idle-timeout to be infinite (#54)

The client was dropping connections to the server when it shouldn't
have. Setting the pool's idle-timeout from the default of 5 seconds to
have no limit resolved this issue.

3 years agoExclude PartitionedIndexedQueryBenchmark. (#53)
Sean Goller [Wed, 13 Feb 2019 18:16:01 +0000 (10:16 -0800)] 
Exclude PartitionedIndexedQueryBenchmark. (#53)

The variance is too large on this.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoExclude tests of Functions
Helena A. Bales [Tue, 12 Feb 2019 23:56:14 +0000 (15:56 -0800)] 
Exclude tests of Functions

3 years agoset -e so errors aren't masked. (#52)
Sean Goller [Mon, 11 Feb 2019 22:24:02 +0000 (14:24 -0800)] 
set -e so errors aren't masked. (#52)

3 years agoGEODE-6340: Missing quotes in bash script.
nabarunnag [Mon, 11 Feb 2019 20:01:33 +0000 (12:01 -0800)] 
GEODE-6340: Missing quotes in bash script.

3 years agoGEODE-6340: add optional metadata to benchmark run (#50)
Helena Bales [Mon, 11 Feb 2019 17:39:48 +0000 (09:39 -0800)] 
GEODE-6340: add optional metadata to benchmark run (#50)

* GEODE-6340: add optional metadata to benchmark run

* add a metadata.json file to the test results
* get metadata from -Pmetadata in gradle command
* add cli option for metadata in run_test and run_against_benchmark
* change benchmark branch cli option from -m to -e
* update cli help message
* add default metadata values including branch names and versions

* update readme and add closing parens to help

* appease spotless

3 years agoGEODE-6332: Add function execution benchmarks (#51)
Nabarun Nag [Fri, 8 Feb 2019 01:50:26 +0000 (17:50 -0800)] 
GEODE-6332: Add function execution benchmarks (#51)

3 years agoGEODE-6311: Adding the OQL benchmarks (#49)
Nabarun Nag [Wed, 23 Jan 2019 21:46:54 +0000 (13:46 -0800)] 
GEODE-6311: Adding the OQL benchmarks (#49)

3 years agoGEODE-6299: Create hgrm files from hlog (#48)
Nabarun Nag [Fri, 18 Jan 2019 22:31:26 +0000 (14:31 -0800)] 
GEODE-6299: Create hgrm files from hlog (#48)

        * Create hgrm files from hlog file created by hdrhistogram.
* This new file is human readable and can be used by plotters to create graphs.

3 years agoAdjust getopt parameters and fix git checkout method. (#47)
Sean Goller [Fri, 18 Jan 2019 17:03:57 +0000 (09:03 -0800)] 
Adjust getopt parameters and fix git checkout method. (#47)

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoAdd debugging because reasons. (#46)
Sean Goller [Fri, 18 Jan 2019 02:17:26 +0000 (18:17 -0800)] 
Add debugging because reasons. (#46)

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoFix defaults and dependencies. (#45)
Sean Goller [Thu, 17 Jan 2019 22:37:46 +0000 (14:37 -0800)] 
Fix defaults and dependencies. (#45)

* Add default for branch under test
* Fetch dependencies before running benchmarks.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoAdd ability to fetch dependencies. (#44)
Sean Goller [Thu, 17 Jan 2019 18:50:24 +0000 (10:50 -0800)] 
Add ability to fetch dependencies. (#44)

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoGEODE-6147 - Fail benchmark task if average latency change is 5+% (#43)
Sean Goller [Thu, 17 Jan 2019 18:35:24 +0000 (10:35 -0800)] 
GEODE-6147 - Fail benchmark task if average latency change is 5+% (#43)

* GEODE-6147 - Fail benchmark task if average latency change is 5+%

* Change ratio to difference
* Add difference calculation to ProbeResult
* Fail analyzeRun task if average latency difference is >= 5%

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoMerge pull request #42 from balesh2/GEODE-6276
Sean Goller [Thu, 17 Jan 2019 17:33:08 +0000 (09:33 -0800)] 
Merge pull request #42 from balesh2/GEODE-6276

GEODE-6276: use named cli options for scripts

3 years agoGEODE-6276: use named cli options for scripts 42/head
Helena A. Bales [Wed, 16 Jan 2019 23:55:11 +0000 (15:55 -0800)] 
GEODE-6276: use named cli options for scripts

Name the cli options for running the benchmark. Support either a version
number or a branch/commit ref. Use version to fetch previously built
releases, to avoid rebuilding them every time.

Signed-off-by: Jacob Barrett <jbarrett@pivotal.io>
3 years agoGEODE-6275: Fail Benchmark job when build fails (#41)
Helena Bales [Tue, 15 Jan 2019 18:52:08 +0000 (10:52 -0800)] 
GEODE-6275: Fail Benchmark job when build fails (#41)

Fail the benchmark job when it fails to build or get resources from
Maven.

Co-authored-by: Jacob Barrett <jbarrett@pivotal.io>
3 years agoAdd public key to authorized_keys on localhost (#39)
Sai Boorlagadda [Mon, 14 Jan 2019 19:57:50 +0000 (11:57 -0800)] 
Add public key to authorized_keys on localhost (#39)

3 years agoGEODE-6134 - Add analysis after running a benchmark comparison. (#40)
Sean Goller [Mon, 14 Jan 2019 19:05:59 +0000 (11:05 -0800)] 
GEODE-6134 - Add analysis after running a benchmark comparison. (#40)

* Add analysis after running a benchmark comparison.

* Add script to run analysis on an output directory
* Call analyze script from benchmark comparison script.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoMerge pull request #38 from apache/aws-script-update
Sean Goller [Fri, 11 Jan 2019 19:02:32 +0000 (11:02 -0800)] 
Merge pull request #38 from apache/aws-script-update

Update script for running tests on AWS

3 years agoUpdate script for running tests on AWS 38/head
Helena A. Bales [Thu, 10 Jan 2019 23:04:12 +0000 (15:04 -0800)] 
Update script for running tests on AWS

Update the script for running tests on AWS to use the default version
number. The gradle parameters for specifying a custom version number
change in different versions. To have one script that works for all
versions, we use the default version number then get that version from
gfsh for use in benchmarks.

Co-Authored-By: jbarrett@pivotal.io
3 years agoAdded information about ssh key-pairs in README.md
nabarunnag [Thu, 10 Jan 2019 14:54:22 +0000 (06:54 -0800)] 
Added information about ssh key-pairs in README.md

3 years agoChanging the test parameters (#37)
Nabarun Nag [Wed, 9 Jan 2019 18:32:37 +0000 (10:32 -0800)] 
Changing the test parameters (#37)

        * Changing the test data package name to be able to run old geode versions
* Test create smaller JVM for travis to be able to run it.

3 years agoIncreasing the the JVM size to 8G
nabarun [Wed, 9 Jan 2019 00:58:00 +0000 (16:58 -0800)] 
Increasing the the JVM size to 8G

3 years agoMerge pull request #36 from smgoller/develop
Sean Goller [Tue, 8 Jan 2019 21:31:33 +0000 (13:31 -0800)] 
Merge pull request #36 from smgoller/develop

Fix credentials management.

3 years agoFix credentials management. 36/head
Jacob Barrett [Tue, 8 Jan 2019 20:32:06 +0000 (12:32 -0800)] 
Fix credentials management.

* If AWS credentials are provided via environment variables, don't set
  the profile.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoMerge pull request #35 from smgoller/develop
Sean Goller [Tue, 8 Jan 2019 18:51:14 +0000 (10:51 -0800)] 
Merge pull request #35 from smgoller/develop

Fixes for grub and TCP syn cookies.

3 years agoFixes for grub and TCP syn cookies. 35/head
Sean Goller [Tue, 8 Jan 2019 18:48:51 +0000 (10:48 -0800)] 
Fixes for grub and TCP syn cookies.

* Make sure 'apt upgrade' can handle grub changes without asking for
help.
* Turn off TCP syn cookies because performance.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoFixing typo in aws configuration instructions
Dan Smith [Fri, 4 Jan 2019 19:47:14 +0000 (11:47 -0800)] 
Fixing typo in aws configuration instructions

3 years agoupdate script parameters in readme (#34)
Helena Bales [Fri, 4 Jan 2019 00:24:47 +0000 (16:24 -0800)] 
update script parameters in readme (#34)

3 years agoAdding additional metrics of standard error
Dan Smith [Thu, 3 Jan 2019 19:44:59 +0000 (11:44 -0800)] 
Adding additional metrics of standard error

Adding additional metrics of standard error

3 years agoUse a dedicated separate profile to work with AWS. (#29)
Sean Goller [Thu, 3 Jan 2019 18:37:00 +0000 (10:37 -0800)] 
Use a dedicated separate profile to work with AWS. (#29)

* Use a dedicated separate profile to work with AWS.

* all scripts now use an AWS profile named `geode-benchmarks`.
* Add script to build AMI.

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoGEODE-6243: New method to get number of members. (#33)
Nabarun Nag [Thu, 3 Jan 2019 18:33:38 +0000 (10:33 -0800)] 
GEODE-6243: New method to get number of members. (#33)

        * New method added to get the number of members for each role.
* Calculating unique addresses for number of members caused an issue when all VMs are in the same physical machine.

3 years agoGEODE-6135: spotless fix for the initial commit.
nabarunnag [Sun, 23 Dec 2018 01:51:59 +0000 (17:51 -0800)] 
GEODE-6135: spotless fix for the initial commit.

3 years agoGEODE-6135 - Changes to support running in CI. 32/head
Sean Goller [Sat, 22 Dec 2018 00:07:40 +0000 (16:07 -0800)] 
GEODE-6135 - Changes to support running in CI.

* Allow output directory to be overridden via commandline in
  run_against_baseline.sh
* Change how branch/tag is checked out in run_tests.sh to support
  SHA references as well
* Minor code cleanup in DestroyCluster class

Authored-by: Sean Goller <sgoller@pivotal.io>
3 years agoGEODE-6233: Use a new hash-map for each PutAll batch in PrePopulateRegion (#31)
Brian Rowe [Fri, 21 Dec 2018 22:10:14 +0000 (14:10 -0800)] 
GEODE-6233: Use a new hash-map for each PutAll batch in PrePopulateRegion (#31)

We see an occasional delay/hang when simply clearing and reusing the same
hashmap for multiple putAll requests.  This change creates a new hashmap every
time it starts a new batch.

This also increases the keyRange for the benchmarks from 10k to 1M.

Co-Authored-By: Helena Bales <hbales@pivotal.io>
3 years agoUsing the dependencies from the local maven repo. (#30)
Nabarun Nag [Thu, 20 Dec 2018 06:29:46 +0000 (22:29 -0800)] 
Using the dependencies from the local maven repo. (#30)

* Using the dependencies from the local maven repo.

* First Apache Geode builds and publishes the dependencies to the local repo
* geode-benchmark will pick up the dependencies from the local repo.

* Adding the output file to gitignore

3 years agoGEODE-6148: improve performance of Prepopulate (#25)
Helena Bales [Fri, 14 Dec 2018 23:51:57 +0000 (15:51 -0800)] 
GEODE-6148: improve performance of Prepopulate (#25)

Prepopulating the region used to occur by means of puts from every
server for every key in the specified range. This resulted in
unnecessarily conflicting puts.

To improve performance of the setup, PrePopulateRegion.run() now divides
the key range as evenly as possible between all servers in the cluster,
prepares the data in a map, then puts the map to the region using
putAll() in threads.

Also:
* add redundancy to Partitioned Regions
* increase JVM Memory

The changes result in an observable improvement in performance.

Signed-off-by: Dan Smith <dsmith@pivotal.io>
3 years agoMerge pull request #27 from smgoller/aws
Sean Goller [Fri, 14 Dec 2018 19:42:13 +0000 (11:42 -0800)] 
Merge pull request #27 from smgoller/aws

Add support for running benchmarks in AWS

3 years agoGEODE-6146: Cleaned Portfolio benchmark object. (#26)
Nabarun Nag [Fri, 14 Dec 2018 06:35:24 +0000 (22:35 -0800)] 
GEODE-6146: Cleaned Portfolio benchmark object. (#26)

* Using automatic reflection based pdx serialization
* Cleaned up the statics in the constructors
* Generics are used.
* Removed unused member variable.

3 years agoRemoving uncessary swap file 27/head
Dan Smith [Fri, 14 Dec 2018 00:24:18 +0000 (16:24 -0800)] 
Removing uncessary swap file

3 years agoReorganizing infrastructure scripts and code
Dan Smith [Fri, 14 Dec 2018 00:22:18 +0000 (16:22 -0800)] 
Reorganizing infrastructure scripts and code

3 years agoAdding command line scripts to run benchmarks in AWS
Dan Smith [Fri, 14 Dec 2018 00:02:21 +0000 (16:02 -0800)] 
Adding command line scripts to run benchmarks in AWS

3 years agoInstalling the ssh key for passwordless ssh
Dan Smith [Thu, 13 Dec 2018 23:17:39 +0000 (15:17 -0800)] 
Installing the ssh key for passwordless ssh