accumulo-testing.git
19 hours agoUpdate cluster.yaml.tftpl (#210) main
Dave Marion [Tue, 5 Jul 2022 11:43:08 +0000 (07:43 -0400)] 
Update cluster.yaml.tftpl (#210)

Remove compaction.compactor.queue element to align with changes made recently in accumulo's main branch

8 days agoBump OpenTelemetry and Micrometer versions (#209)
Dave Marion [Mon, 27 Jun 2022 11:07:05 +0000 (07:07 -0400)] 
Bump OpenTelemetry and Micrometer versions (#209)

3 weeks agoUse ZK 3.5.10 by default
Christopher Tubbs [Wed, 8 Jun 2022 21:22:27 +0000 (17:22 -0400)] 
Use ZK 3.5.10 by default

* Use ZK 3.5.10, to be consistent with the current default Accumulo
  version
* Update parent POM version, and formatter/impsort plugins
* Update NOTICE for current copyright year

7 weeks agoTerraform config fixes based on testing (#207)
Brian Loss [Thu, 12 May 2022 11:42:28 +0000 (07:42 -0400)] 
Terraform config fixes based on testing (#207)

* Terraform config fixes based on testing

* Update AzureRM Terraform provider to use any 3.x version.
* Fix bug where variable validation would fail for the Azure
  configuration if managed_disk_configuration was not used and set to
  null.
* Clarify in the README that terraform init/plan/apply commands are
  sensitive to the current working directory.
* Clarify in the README that the appropriate SSH private key must be
  present in the SSH agent in order for terraform apply to succeed.
* Fix step numbering in QUICKSTART
* Remove sentence fragment about creating config from step 4 of
  QUICKSTART
* Add note about needing the specified ssh key to be loaded in your
  ssh-agent for provisioning to succeed

8 weeks agoruns mvn install for accumulo in terraform (#206)
Keith Turner [Tue, 10 May 2022 11:03:13 +0000 (07:03 -0400)] 
runs mvn install for accumulo in terraform (#206)

Run mvn install when building accumulo so that the accumulo-testing will pick up the correct accumulo snapshot jars.

8 weeks agoDrop Replication concurrent RW test and IT (#205)
Mike Miller [Mon, 9 May 2022 18:00:03 +0000 (14:00 -0400)] 
Drop Replication concurrent RW test and IT (#205)

* Closes #171

2 months agoManaged disk support for Azure terraform infra (#202)
Brian Loss [Thu, 21 Apr 2022 13:01:44 +0000 (09:01 -0400)] 
Managed disk support for Azure terraform infra (#202)

Support adding managed disk to the Azure VMs created by the terraform
testing infrastructure. By adding multiple managed disks to a VM, we can
get significantly more space for data storage and also increase
performance since the data is striped across multiple disks.

* Modify the cloud-init module to accept an argument indicating the type
  of deployment (AWS or Azure) so that conditional blocks can be
  included in the cloud-init script.
* cloud-init module now accepts an optional lvm_mount_point argument. If
  this argument is specified, then the cloud-init script will assume
  that managed disks were created and load a script on the VM and run it
  to wait for the disks to be attached, then group them in an LVM volume
  that is mounted under the specified mount point.
* The azure main.tf file accepts a new managed_disk_configuration
  optional argument that contains the LVM mount point, and the number,
  size, and sku of managed disks to add to each VM. If this argument is
  specified, then the managed disks are created and attached to the VMs,
  and the lvm mount point and expected number of disks are passed along
  to the cloud-init module. Due to the way attaching managed disks are
  supported by Terraform (they must be attached after the VM is created,
  although Azure does not have this restriction), the provisioner script
  that waits for cloud-init to complete had to be moved outside of the
  VM creation to a null_resource. This null_resource must then be
  explicitly added as a dependency of any module that requires the
  manager or worker VMs to be created AND have cloud-init completed
  running.
* Fix bug in Azure configuration where the script would fail if the
  create_resource_group variable was set to false (indicating that an
  existing resource group should be used instead of creating a new one).
* Update the maven version to 3.8.5.

2 months agoUpdate OpenTelemetry and Micrometer versions to match Accumulo 2.1.0 (#201)
Dave Marion [Thu, 7 Apr 2022 18:11:58 +0000 (14:11 -0400)] 
Update OpenTelemetry and Micrometer versions to match Accumulo 2.1.0 (#201)

3 months agoWait for the NameNode to leave safemode before init'ing Accumulo (#200)
Dave Marion [Mon, 4 Apr 2022 15:54:24 +0000 (11:54 -0400)] 
Wait for the NameNode to leave safemode before init'ing Accumulo (#200)

Co-authored-by: Brian Loss <brianloss@gmail.com>
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
3 months agoMigrate all tests to JUnit5 (#198)
Dom G [Fri, 25 Mar 2022 13:54:51 +0000 (09:54 -0400)] 
Migrate all tests to JUnit5 (#198)

3 months agoUse non deprecated version of Hashing.murmur3_32() (#199)
Dom G [Thu, 24 Mar 2022 18:33:04 +0000 (14:33 -0400)] 
Use non deprecated version of Hashing.murmur3_32() (#199)

3 months agoFixup testing for changes in Accumulo regarding ThreadPools (#197)
Dave Marion [Thu, 17 Mar 2022 17:03:43 +0000 (13:03 -0400)] 
Fixup testing for changes in Accumulo regarding ThreadPools (#197)

4 months agoupgrade versioning in performance script (#195)
Jeffrey Manno [Sun, 27 Feb 2022 23:57:54 +0000 (18:57 -0500)] 
upgrade versioning in performance script (#195)

4 months agoUse `NewTableConfiguration().withSplits()` instead of `addSplits()` (#193)
Dom G [Tue, 22 Feb 2022 14:08:02 +0000 (09:08 -0500)] 
Use `NewTableConfiguration().withSplits()` instead of `addSplits()` (#193)

* Convert to use NTC

4 months agoFix broken links in readme (#194)
Dom G [Fri, 18 Feb 2022 20:02:05 +0000 (15:02 -0500)] 
Fix broken links in readme (#194)

Fix broken links to modules directory and Image.xml file within randomwalk/

4 months agoMake MulitTable reuse tables (#191)
Mike Miller [Fri, 18 Feb 2022 16:47:29 +0000 (11:47 -0500)] 
Make MulitTable reuse tables (#191)

* This allows restarting rw tests and allow using of the same tables
across multiple threads
* Other various improvements to MultiTable

4 months agoTerraform configurations for creating test infrastructure (#185)
Dave Marion [Mon, 14 Feb 2022 20:05:21 +0000 (15:05 -0500)] 
Terraform configurations for creating test infrastructure (#185)

This change includes Terraform configurations for creating and destroying compute resources for testing on AWS and Azure. The configurations install ZooKeeper, Hadoop, Accumulo, and Accumulo-Testing. Users can supply options for the versions of the software that should be installed or can supply their own binary tarballs for installation. See the README for detailed documentation.

Co-authored-by: Brian Loss <brianloss@gmail.com>
Co-authored-by: domgarguilo <dominic.garguilo@gmail.com>
4 months agoUpdate usages of ThreadPools.createFixedThreadPool to include new parameter (#189)
Dom G [Fri, 4 Feb 2022 19:08:48 +0000 (14:08 -0500)] 
Update usages of ThreadPools.createFixedThreadPool to include new parameter (#189)

5 months agoEliminate possible race condition in `agitator stop` script (#187)
Dom G [Wed, 19 Jan 2022 14:11:22 +0000 (09:11 -0500)] 
Eliminate possible race condition in `agitator stop` script (#187)

* Kill all processes in agitators proccess group
* Improve log msg

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
5 months agoMinor improvements to agitator readme (#186)
Dom G [Thu, 13 Jan 2022 18:45:26 +0000 (13:45 -0500)] 
Minor improvements to agitator readme (#186)

5 months agoFix broken agitator scripts (#184)
Mike Miller [Wed, 12 Jan 2022 20:28:01 +0000 (15:28 -0500)] 
Fix broken agitator scripts (#184)

* Create bash function to start agitators
* Drop broken perl scripts in favor of bash functions
* Add function to call ClusterConfigParser to read cluster.yaml
* Make agitators only log to one file
* Rename env variables from master to manager

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
Co-authored-by: Dom G. <domgarguilo@apache.org>
5 months agoExport PSSH (followup to #179)
Christopher Tubbs [Wed, 5 Jan 2022 20:19:12 +0000 (15:19 -0500)] 
Export PSSH (followup to #179)

Follow-up fix to export PSSH for agitator

The PSSH environment variable needs to be exported by bin/agitator so it
can be seen inside the libexec/master-agitator.pl script.

6 months agoReplace use of `pssh` with `parallel-ssh` (#179)
Dom G [Thu, 16 Dec 2021 19:39:28 +0000 (14:39 -0500)] 
Replace use of `pssh` with `parallel-ssh` (#179)

* Replaced pssh and pscp
* Fall back to parallel-ssh when pssh unavailable
* Fix quoting in sudo commands in agitator
* Preserve PSSH environment over sudo commands for agitator
* Properly set PSSH in perl agitator code from process environment
* Fix a couple typos in READMEs

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
6 months agoUse accumulo-project for dependency mgmt (#181)
Christopher Tubbs [Wed, 15 Dec 2021 17:34:10 +0000 (12:34 -0500)] 
Use accumulo-project for dependency mgmt (#181)

Fix warnings about dependency versions by using the accumulo-project POM
to manage dependencies. This alleviates the need to keep dependency
versions sync'd with the Accumulo main project. Also fix a few other
warnings throughout the project.

* Update POM to make accumulo-project pom manage the dependency versions
  for this project (for dependencies used by that pom, anyway)
* Remove unnecessary dependency version management in this project's pom
* Remove log4j direct usage in ClientOpts and ContinuousOpts (log4j2
  properties file can be used to control logging debug/trace levels
  instead, dynamically if desired)
* Use commons-lang3 where possible, since commons-lang is no longer on
  the class path
* Switch out deprecated properties with their replacement, when there is
  a direct replacement
* Update TimedScanDispatcher to use the dispatch name with correctly
  spelled parameters
* Inline XMLSchema constants to avoid problems with
  javax.xml.XMLConstants being ambiguously grabbed from a jar on the
  class path
* Remove use of TABLE_MAJC_COMPACTALL_IDLETIME compaction property,
  since it was deprecated without replacement, for table settings in the
  randomwalk config
* Suppress deprecation warnings by deprecating the Replication
  randomwalk class
* Replace finalize method in DataWriter with a Cleaner

6 months agofix javadoc warnings (#178)
EdColeman [Wed, 15 Dec 2021 14:39:56 +0000 (09:39 -0500)] 
fix javadoc warnings (#178)

Co-authored-by: Ed Coleman etcoleman <edcoleman@apache.org>
Co-authored-by: Dom G. <47725857+DomGarguilo@users.noreply.github.com>
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
6 months agoBump parent POM and plugin versions (#180)
Christopher Tubbs [Wed, 15 Dec 2021 14:03:26 +0000 (09:03 -0500)] 
Bump parent POM and plugin versions (#180)

Update pom.xml with updated parent POM and newer plugin versions
Add javadoc rules to stop warnings on missing unnecessary standard tags

6 months agoContinuous Ingest improvements (#168)
Dom G [Wed, 8 Dec 2021 15:04:43 +0000 (10:04 -0500)] 
Continuous Ingest improvements (#168)

* consolidated for-loops
* improved logs
* avoided parsing properties multiple times
* move AccumuloClient and BatchWriter into try-with-resources
* created member variables to reuse values & simplify methods
* renamed variables and made some final

6 months agoSome minor code cleanup throughout accumulo-testing (#176)
Jeffrey Manno [Tue, 7 Dec 2021 15:02:12 +0000 (10:02 -0500)] 
Some minor code cleanup throughout accumulo-testing (#176)

7 months agoUpdate zookeeper version to fix Overlapping warning (#174)
Jeffrey Manno [Tue, 30 Nov 2021 16:49:53 +0000 (11:49 -0500)] 
Update zookeeper version to fix Overlapping warning (#174)

7 months agoFix build warnings in a few classes due to depreciation
Jeffrey Manno [Wed, 24 Nov 2021 22:33:00 +0000 (17:33 -0500)] 
Fix build warnings in a few classes due to depreciation

7 months agoFix module-info warnings
Jeffrey Manno [Wed, 24 Nov 2021 22:31:34 +0000 (17:31 -0500)] 
Fix module-info warnings

7 months agoformat 172/head
Jeffrey Manno [Wed, 24 Nov 2021 22:31:10 +0000 (17:31 -0500)] 
format

7 months agoUpdate src/main/java/org/apache/accumulo/testing/performance/impl/PerfTestRunner...
Jeffrey Manno [Wed, 24 Nov 2021 22:29:31 +0000 (17:29 -0500)] 
Update src/main/java/org/apache/accumulo/testing/performance/impl/PerfTestRunner.java

Co-authored-by: Mike Miller <mmiller@apache.org>
7 months agoremove overlapping fix due to bug with perf test 173/head
Jeffrey Manno [Wed, 24 Nov 2021 19:02:31 +0000 (14:02 -0500)] 
remove overlapping fix due to bug with perf test

7 months agoone more
Jeffrey Manno [Wed, 24 Nov 2021 19:01:19 +0000 (14:01 -0500)] 
one more

7 months agoFix zookeeper overlapping classes and module-info warnings
Jeffrey Manno [Wed, 24 Nov 2021 18:28:19 +0000 (13:28 -0500)] 
Fix zookeeper overlapping classes and module-info warnings

7 months agoFix build warnings in a few classes due to deprication
Jeffrey Manno [Wed, 24 Nov 2021 18:10:23 +0000 (13:10 -0500)] 
Fix build warnings in a few classes due to deprication

7 months agoUse correct createFixedThreadPool function
Jeffrey Manno [Tue, 23 Nov 2021 20:06:36 +0000 (15:06 -0500)] 
Use correct createFixedThreadPool function

7 months agoUse correct createFixedThreadPool function 170/head
Jeffrey Manno [Tue, 23 Nov 2021 19:49:46 +0000 (14:49 -0500)] 
Use correct createFixedThreadPool function

7 months agoAdd deletes to continuous ingest (#166)
Dom G [Fri, 19 Nov 2021 18:04:02 +0000 (13:04 -0500)] 
Add deletes to continuous ingest (#166)

* Add the ability for the deletion of entries to occur while running continuous ingest

8 months agoIgnore ReplicaitonRandomWalkIT from maven verify build
Jeffrey Manno [Tue, 19 Oct 2021 21:40:30 +0000 (17:40 -0400)] 
Ignore ReplicaitonRandomWalkIT from maven verify build

8 months agoignore replication IT inside accumulo-testing 165/head
Jeffrey Manno [Tue, 19 Oct 2021 17:24:38 +0000 (13:24 -0400)] 
ignore replication IT inside accumulo-testing

8 months agoFix failing ScanExecutorPT (#163)
Dom G [Tue, 5 Oct 2021 14:30:11 +0000 (10:30 -0400)] 
Fix failing ScanExecutorPT (#163)

* replaced outdated hinting

Co-authored-by: Keith Turner <kturner@apache.org>
Co-authored-by: Jeffrey Manno <jeffreymanno15@gmail.com>
9 months agoUpdate readme and add comments to RandomCachedLookupsPT
Jeffrey Manno [Wed, 29 Sep 2021 13:08:20 +0000 (09:08 -0400)] 
Update readme and add comments to RandomCachedLookupsPT

* These additions aim to inform the user that some tests may require changes to certain property values in order to run.

9 months agoAdd documentation, comments, and warning fixes 160/head
Jeffrey Manno [Tue, 28 Sep 2021 21:26:40 +0000 (17:26 -0400)] 
Add documentation, comments, and warning fixes

9 months agoAdd units to performance test results (#156)
Dom G [Fri, 24 Sep 2021 15:23:03 +0000 (11:23 -0400)] 
Add units to performance test results (#156)

* Add units to PT output
* Added constants and corrections
* Updated results descriptions

9 months agoFix performance test NumberFormatException bug (#155)
Dom G [Thu, 23 Sep 2021 20:25:18 +0000 (16:25 -0400)] 
Fix performance test NumberFormatException bug (#155)

* Fixed NumberFormatException
* Add constants, make runs consistent
* Make var names more meaningful

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
9 months agoAdd example of a single performance test run (#158)
Dom G [Thu, 23 Sep 2021 14:04:54 +0000 (10:04 -0400)] 
Add example of a single performance test run (#158)

* Update readme to include single performance test run example
* Misc. small grammar fixes

9 months agoMigrate accumulo-testing to log4j2. (#146)
AlbertWhitlock [Mon, 20 Sep 2021 18:12:12 +0000 (14:12 -0400)] 
Migrate accumulo-testing to log4j2. (#146)

Migrate accumulo-testing to use log4j2 (via slf4j)

Co-authored-by: Jeffrey Manno <jeffreymanno15@gmail.com>
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
9 months agoCreate unique directory for each run of `cingest verify` to store test output (#152)
Dom G [Fri, 17 Sep 2021 16:01:09 +0000 (12:01 -0400)] 
Create unique directory for each run of `cingest verify` to store test output (#152)

* Now creates a new unique directory to store test output of each run of `cingest verify
* Now no longer stops the test when the output directory already exists

9 months agoreplace use of deprecated newInstance() (#151)
Dom G [Fri, 10 Sep 2021 19:08:52 +0000 (15:08 -0400)] 
replace use of deprecated newInstance() (#151)

Replaced the use of the deprecated newInstance() method with its replacement getDeclaredConstructor().newInstance()

9 months agoRefactor out Non API code (#150)
Dom G [Fri, 10 Sep 2021 19:06:59 +0000 (15:06 -0400)] 
Refactor out Non API code (#150)

Removed or replaced usages of non-API code

10 months agoMerge pull request #149 from DomGarguilo/updateCI
Dom G [Thu, 2 Sep 2021 12:25:00 +0000 (08:25 -0400)] 
Merge pull request #149 from DomGarguilo/updateCI

* Add capability to change compaction planner/dispatcher
* Refactored prop name
* Remove external comps

10 months agoRemove external comps. and replace foreach 149/head
domgarguilo [Mon, 23 Aug 2021 15:07:11 +0000 (11:07 -0400)] 
Remove external comps. and replace foreach

10 months agoRefactored prop name
domgarguilo [Tue, 17 Aug 2021 17:53:07 +0000 (13:53 -0400)] 
Refactored prop name

10 months agoAdd capability to change compaction planner/dispatcher
domgarguilo [Tue, 17 Aug 2021 17:44:04 +0000 (13:44 -0400)] 
Add capability to change compaction planner/dispatcher

10 months agoMerge pull request #143 from apache/update_guava_version_30_1_1
EdColeman [Mon, 16 Aug 2021 21:48:00 +0000 (17:48 -0400)] 
Merge pull request #143 from apache/update_guava_version_30_1_1

Update guava version to 30.1.1 (matches accumulo project)

10 months agoUpdate guava version (rebased to current main) 143/head
Ed Coleman etcoleman [Mon, 16 Aug 2021 21:31:02 +0000 (21:31 +0000)] 
Update guava version (rebased to current main)

10 months agoReplace deprecated prop inside ReplicationRandomWalkIT with its replacement
Jeffrey Manno [Mon, 9 Aug 2021 16:06:26 +0000 (12:06 -0400)] 
Replace deprecated prop inside ReplicationRandomWalkIT with its replacement

10 months agoReplace deprecated prop inside ReplicationRandomWalkIT 147/head
Jeffrey Manno [Mon, 9 Aug 2021 15:42:43 +0000 (11:42 -0400)] 
Replace deprecated prop inside ReplicationRandomWalkIT

13 months agoProperly build shaded jar when running cingest (#140)
AlbertWhitlock [Tue, 1 Jun 2021 14:39:09 +0000 (10:39 -0400)] 
Properly build shaded jar when running cingest (#140)

14 months agoUpdate pom to Java 11 (#138)
Mike Miller [Wed, 14 Apr 2021 11:53:38 +0000 (07:53 -0400)] 
Update pom to Java 11 (#138)

* Update pom to Java 11
* Remove obsoleted profile
* Use java 11 for m2e profile as well

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
15 months agoBump guava from 26.0-jre to 29.0-jre (#137)
dependabot[bot] [Thu, 1 Apr 2021 21:42:39 +0000 (17:42 -0400)] 
Bump guava from 26.0-jre to 29.0-jre (#137)

Bumps [guava](https://github.com/google/guava) from 26.0-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
16 months agoFix typo in bin/cingest script
Mike Miller [Tue, 23 Feb 2021 13:28:57 +0000 (08:28 -0500)] 
Fix typo in bin/cingest script

16 months agoAdd back END to MultiTable so it can be used by other modules
Mike Miller [Mon, 22 Feb 2021 19:40:12 +0000 (14:40 -0500)] 
Add back END to MultiTable so it can be used by other modules

16 months agoAdd more splits to RW multitable CreateTable
Mike Miller [Wed, 17 Feb 2021 14:21:10 +0000 (09:21 -0500)] 
Add more splits to RW multitable CreateTable

16 months agoFixes #135. Fix build errors due recent changes in accumulo and upgrade accumulo...
Jeffrey Manno [Thu, 11 Feb 2021 18:40:30 +0000 (13:40 -0500)] 
Fixes #135. Fix build errors due recent changes in accumulo and upgrade accumulo version to 2.1.0

16 months agoupgrade accumulo to 2.1.0-SNAPSHOT 136/head
Jeffrey Manno [Thu, 11 Feb 2021 14:27:39 +0000 (09:27 -0500)] 
upgrade accumulo to 2.1.0-SNAPSHOT

16 months agoadd import class exception for ThreadPools
Jeffrey Manno [Thu, 11 Feb 2021 14:24:58 +0000 (09:24 -0500)] 
add import class exception for ThreadPools

16 months agofix build errors
Jeffrey Manno [Thu, 11 Feb 2021 13:37:49 +0000 (08:37 -0500)] 
fix build errors

16 months agoAdd a Bulk import to randomwalk MultiTable (#134)
Mike Miller [Tue, 9 Feb 2021 21:48:55 +0000 (16:48 -0500)] 
Add a Bulk import to randomwalk MultiTable (#134)

* Create new BulkImport test in MultiTable for more realistic case
* Tweak MultiTable.xml to drop 20 tables, same as create. This will keep
the number of talbes down when running for long period.
* Also set log4j testing logging to DEBUG

17 months agoExclude log4j2 jars from shade plugin (#133)
Mike Miller [Fri, 8 Jan 2021 13:15:28 +0000 (08:15 -0500)] 
Exclude log4j2 jars from shade plugin (#133)

17 months agoFix warning in randomwalk Init
Mike Miller [Tue, 5 Jan 2021 12:48:13 +0000 (07:48 -0500)] 
Fix warning in randomwalk Init

* Fix redundant cast to java.util.Random warning in randomwalk Init

18 months agoupdate test with correct API call (#131)
rchoksi-hw [Mon, 14 Dec 2020 15:09:04 +0000 (07:09 -0800)] 
update test with correct API call (#131)

Co-authored-by: romil choksi <rchoksi@cloudera.com>
20 months agoUpload test logs on build failure
Christopher Tubbs [Tue, 13 Oct 2020 02:04:15 +0000 (22:04 -0400)] 
Upload test logs on build failure

20 months agoBump junit from 4.12 to 4.13.1 (#129)
dependabot[bot] [Tue, 13 Oct 2020 01:13:43 +0000 (21:13 -0400)] 
Bump junit from 4.12 to 4.13.1 (#129)

Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
20 months agoAdd .asf.yaml (#128)
Christopher Tubbs [Mon, 5 Oct 2020 21:16:30 +0000 (17:16 -0400)] 
Add .asf.yaml (#128)

20 months agoMerge pull request #125 from keith-turner/gcpt
Keith Turner [Mon, 5 Oct 2020 16:54:45 +0000 (12:54 -0400)] 
Merge pull request #125 from keith-turner/gcpt

Add group commit performance test

21 months agoremoves uneeded import from performance script (#126)
Keith Turner [Mon, 28 Sep 2020 17:59:53 +0000 (13:59 -0400)] 
removes uneeded import from performance script (#126)

21 months agopresplit table 125/head
Keith Turner [Sat, 26 Sep 2020 01:18:33 +0000 (21:18 -0400)] 
presplit table

21 months agofixes #67 improves group commit performance test
Keith Turner [Fri, 25 Sep 2020 19:14:33 +0000 (15:14 -0400)] 
fixes #67 improves group commit performance test

21 months agoFix #67: Test group commit performance
Elina Wise [Fri, 24 May 2019 16:25:11 +0000 (12:25 -0400)] 
Fix #67: Test group commit performance

21 months agoMissing license headers #122 (#124)
Mark Owens [Tue, 22 Sep 2020 20:02:08 +0000 (16:02 -0400)] 
Missing license headers #122 (#124)

* Added missing license headers to 20+ files.
* Added the apache-rat plugin to the pom file.
* Moved the splits data files in test/bench/lib into a data
subdirectory.
* Modified TableSplitsBenchmark.py to account for the movement of the
splits file into the data directory.
* Added exclusions for the data directory and the exlude_classes.txt
file.
* Update .gitignore to ignore .pyc files.

22 months agoUpdate for main branch and GitHub Actions CI
Christopher Tubbs [Tue, 11 Aug 2020 13:27:41 +0000 (09:27 -0400)] 
Update for main branch and GitHub Actions CI

2 years agoUpdate README.md
Mark Owens [Wed, 27 May 2020 16:41:12 +0000 (12:41 -0400)] 
Update README.md

While reading over the accumulo-testing readme, I noticed a couple of typos. This update corrects the ones I found.

2 years agoAdding Accumulo Availability Monitor :
Tushar D [Thu, 14 Nov 2019 02:17:32 +0000 (18:17 -0800)] 
Adding Accumulo Availability Monitor :

Monitor.Java : This file includes core code for scan operation that randomly picks a tablet server, identifies its minimum and maximum rows range and picks out 'distance' (configurable through opts) number of rows to be scanned in one iteration.

MonitorOpts.java : This file is used to provide all the parameters which are taken by the Monitor Class. Interesting Options Include : Ability to set scanner sleep time , between iterations. Setting Batch Size that scanner uses internally. Setting number of iteration or continuous mode.

monitor script : This follows same pattern as cingest, rwalk, performance scripts.

Caching Splits

Schema Agnostic Healthprobe

Added Changes to make the healthprobe schema agnostic, while retaining the functionality to scan configurable number of rows. Rows are scanned from beginning of the tablet. Tablet information is loaded once and cached for later iterations.

Signed-off-by: Marc Parisi <phrocker@apache.org>
2 years agoFix #120 Support non default/absolute hdfs paths in continuous bulk ingestion (#121)
Mandar Inamdar [Mon, 2 Dec 2019 14:59:22 +0000 (07:59 -0700)] 
Fix #120 Support non default/absolute hdfs paths in continuous bulk ingestion (#121)

* Accept non default/absolute hdfs paths too as a parameter to cingest bulk
* Without this change, only defaultFS works
* Without this change, it gives error for any parameter other than relative path
* This feature support is critical, especially in case of multiple hdfs volumes
* Consistent usage of fs.makeQualified and reusing existing bulkDir variable

2 years agoFix #37 Restart GC during agitation (#117)
billierinaldi [Tue, 12 Nov 2019 21:52:39 +0000 (13:52 -0800)] 
Fix #37 Restart GC during agitation (#117)

2 years agoMove configuration of checkstyle plugin
Mike Miller [Thu, 7 Nov 2019 20:15:14 +0000 (15:15 -0500)] 
Move configuration of checkstyle plugin

2 years agoRemove explicit checkstyle dependency
Mike Miller [Wed, 6 Nov 2019 22:59:16 +0000 (17:59 -0500)] 
Remove explicit checkstyle dependency

2 years agoSupport extra Java options (#116)
Arvind Shyamsundar [Mon, 28 Oct 2019 22:32:32 +0000 (15:32 -0700)] 
Support extra Java options (#116)

Interjects an extra $JAVA_OPTS environemnt variable wherever Java is
being launched. This adds flexibility for the user to define extra
options, for example Xmx.

2 years agoFixes issue #114, incorrect auths used by scanners and walkers (#115)
Mandar Inamdar [Mon, 7 Oct 2019 23:44:11 +0000 (16:44 -0700)] 
Fixes issue #114, incorrect auths used by scanners and walkers (#115)

Fixed authValue.split("|") call in ContinuousEnv.java.
It was not considering | as a metacharacter in regex used by split
added escape character \\ in the split function call as authValue.split("\\|")

2 years agoREADME.md for automateEC2.sh (#113)
Russell Carter [Thu, 26 Sep 2019 20:05:31 +0000 (16:05 -0400)] 
README.md for automateEC2.sh  (#113)

2 years agoResolves #78 Script to automate EC2 (#96)
Russell Carter [Fri, 20 Sep 2019 16:09:52 +0000 (12:09 -0400)] 
Resolves #78 Script to automate EC2  (#96)

2 years agoUpdate ContinuousInputFormat to use col vis
Mike Miller [Thu, 19 Sep 2019 22:33:26 +0000 (18:33 -0400)] 
Update ContinuousInputFormat to use col vis

2 years agoUpdate cluster-control.sh.uno with uno updates
Mike Miller [Tue, 17 Sep 2019 20:52:52 +0000 (16:52 -0400)] 
Update cluster-control.sh.uno with uno updates

2 years agoAdd check for Test class to peformance script
Mike Miller [Tue, 17 Sep 2019 20:23:30 +0000 (16:23 -0400)] 
Add check for Test class to peformance script

2 years agoFixes #59 Create herding performance test (#93)
Laura Schanno [Tue, 17 Sep 2019 19:26:10 +0000 (15:26 -0400)] 
Fixes #59 Create herding performance test (#93)

* Add HeardingPT.java in order to test herding performance.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.

Related: #53

2 years agoMove performance tests to accumulo-testing #1200 (#90)
Laura Schanno [Tue, 17 Sep 2019 15:45:25 +0000 (11:45 -0400)] 
Move performance tests to accumulo-testing #1200 (#90)

* Move ManySplitIT as HighSplitCreationPT.
* Move BalanceFasterIT as SplitBalancingPT.
* Move DeleteTableDuringSplitIT as TableDeletionDuringSplitPT.
* Move RollWALPerformanceIT as RollWALPT.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.

Related: #1200

2 years agoFix BulkImport concurrent rwalk
Mike Miller [Thu, 12 Sep 2019 17:06:55 +0000 (13:06 -0400)] 
Fix BulkImport concurrent rwalk