accumulo-testing.git
10 months agoReplace deprecated prop inside ReplicationRandomWalkIT 147/head
Jeffrey Manno [Mon, 9 Aug 2021 15:42:43 +0000 (11:42 -0400)] 
Replace deprecated prop inside ReplicationRandomWalkIT

13 months agoProperly build shaded jar when running cingest (#140)
AlbertWhitlock [Tue, 1 Jun 2021 14:39:09 +0000 (10:39 -0400)] 
Properly build shaded jar when running cingest (#140)

14 months agoUpdate pom to Java 11 (#138)
Mike Miller [Wed, 14 Apr 2021 11:53:38 +0000 (07:53 -0400)] 
Update pom to Java 11 (#138)

* Update pom to Java 11
* Remove obsoleted profile
* Use java 11 for m2e profile as well

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
15 months agoBump guava from 26.0-jre to 29.0-jre (#137)
dependabot[bot] [Thu, 1 Apr 2021 21:42:39 +0000 (17:42 -0400)] 
Bump guava from 26.0-jre to 29.0-jre (#137)

Bumps [guava](https://github.com/google/guava) from 26.0-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
16 months agoFix typo in bin/cingest script
Mike Miller [Tue, 23 Feb 2021 13:28:57 +0000 (08:28 -0500)] 
Fix typo in bin/cingest script

16 months agoAdd back END to MultiTable so it can be used by other modules
Mike Miller [Mon, 22 Feb 2021 19:40:12 +0000 (14:40 -0500)] 
Add back END to MultiTable so it can be used by other modules

16 months agoAdd more splits to RW multitable CreateTable
Mike Miller [Wed, 17 Feb 2021 14:21:10 +0000 (09:21 -0500)] 
Add more splits to RW multitable CreateTable

16 months agoFixes #135. Fix build errors due recent changes in accumulo and upgrade accumulo...
Jeffrey Manno [Thu, 11 Feb 2021 18:40:30 +0000 (13:40 -0500)] 
Fixes #135. Fix build errors due recent changes in accumulo and upgrade accumulo version to 2.1.0

16 months agoupgrade accumulo to 2.1.0-SNAPSHOT 136/head
Jeffrey Manno [Thu, 11 Feb 2021 14:27:39 +0000 (09:27 -0500)] 
upgrade accumulo to 2.1.0-SNAPSHOT

16 months agoadd import class exception for ThreadPools
Jeffrey Manno [Thu, 11 Feb 2021 14:24:58 +0000 (09:24 -0500)] 
add import class exception for ThreadPools

16 months agofix build errors
Jeffrey Manno [Thu, 11 Feb 2021 13:37:49 +0000 (08:37 -0500)] 
fix build errors

16 months agoAdd a Bulk import to randomwalk MultiTable (#134)
Mike Miller [Tue, 9 Feb 2021 21:48:55 +0000 (16:48 -0500)] 
Add a Bulk import to randomwalk MultiTable (#134)

* Create new BulkImport test in MultiTable for more realistic case
* Tweak MultiTable.xml to drop 20 tables, same as create. This will keep
the number of talbes down when running for long period.
* Also set log4j testing logging to DEBUG

17 months agoExclude log4j2 jars from shade plugin (#133)
Mike Miller [Fri, 8 Jan 2021 13:15:28 +0000 (08:15 -0500)] 
Exclude log4j2 jars from shade plugin (#133)

18 months agoFix warning in randomwalk Init
Mike Miller [Tue, 5 Jan 2021 12:48:13 +0000 (07:48 -0500)] 
Fix warning in randomwalk Init

* Fix redundant cast to java.util.Random warning in randomwalk Init

18 months agoupdate test with correct API call (#131)
rchoksi-hw [Mon, 14 Dec 2020 15:09:04 +0000 (07:09 -0800)] 
update test with correct API call (#131)

Co-authored-by: romil choksi <rchoksi@cloudera.com>
20 months agoUpload test logs on build failure
Christopher Tubbs [Tue, 13 Oct 2020 02:04:15 +0000 (22:04 -0400)] 
Upload test logs on build failure

20 months agoBump junit from 4.12 to 4.13.1 (#129)
dependabot[bot] [Tue, 13 Oct 2020 01:13:43 +0000 (21:13 -0400)] 
Bump junit from 4.12 to 4.13.1 (#129)

Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
21 months agoAdd .asf.yaml (#128)
Christopher Tubbs [Mon, 5 Oct 2020 21:16:30 +0000 (17:16 -0400)] 
Add .asf.yaml (#128)

21 months agoMerge pull request #125 from keith-turner/gcpt
Keith Turner [Mon, 5 Oct 2020 16:54:45 +0000 (12:54 -0400)] 
Merge pull request #125 from keith-turner/gcpt

Add group commit performance test

21 months agoremoves uneeded import from performance script (#126)
Keith Turner [Mon, 28 Sep 2020 17:59:53 +0000 (13:59 -0400)] 
removes uneeded import from performance script (#126)

21 months agopresplit table 125/head
Keith Turner [Sat, 26 Sep 2020 01:18:33 +0000 (21:18 -0400)] 
presplit table

21 months agofixes #67 improves group commit performance test
Keith Turner [Fri, 25 Sep 2020 19:14:33 +0000 (15:14 -0400)] 
fixes #67 improves group commit performance test

21 months agoFix #67: Test group commit performance
Elina Wise [Fri, 24 May 2019 16:25:11 +0000 (12:25 -0400)] 
Fix #67: Test group commit performance

21 months agoMissing license headers #122 (#124)
Mark Owens [Tue, 22 Sep 2020 20:02:08 +0000 (16:02 -0400)] 
Missing license headers #122 (#124)

* Added missing license headers to 20+ files.
* Added the apache-rat plugin to the pom file.
* Moved the splits data files in test/bench/lib into a data
subdirectory.
* Modified TableSplitsBenchmark.py to account for the movement of the
splits file into the data directory.
* Added exclusions for the data directory and the exlude_classes.txt
file.
* Update .gitignore to ignore .pyc files.

22 months agoUpdate for main branch and GitHub Actions CI
Christopher Tubbs [Tue, 11 Aug 2020 13:27:41 +0000 (09:27 -0400)] 
Update for main branch and GitHub Actions CI

2 years agoUpdate README.md
Mark Owens [Wed, 27 May 2020 16:41:12 +0000 (12:41 -0400)] 
Update README.md

While reading over the accumulo-testing readme, I noticed a couple of typos. This update corrects the ones I found.

2 years agoAdding Accumulo Availability Monitor :
Tushar D [Thu, 14 Nov 2019 02:17:32 +0000 (18:17 -0800)] 
Adding Accumulo Availability Monitor :

Monitor.Java : This file includes core code for scan operation that randomly picks a tablet server, identifies its minimum and maximum rows range and picks out 'distance' (configurable through opts) number of rows to be scanned in one iteration.

MonitorOpts.java : This file is used to provide all the parameters which are taken by the Monitor Class. Interesting Options Include : Ability to set scanner sleep time , between iterations. Setting Batch Size that scanner uses internally. Setting number of iteration or continuous mode.

monitor script : This follows same pattern as cingest, rwalk, performance scripts.

Caching Splits

Schema Agnostic Healthprobe

Added Changes to make the healthprobe schema agnostic, while retaining the functionality to scan configurable number of rows. Rows are scanned from beginning of the tablet. Tablet information is loaded once and cached for later iterations.

Signed-off-by: Marc Parisi <phrocker@apache.org>
2 years agoFix #120 Support non default/absolute hdfs paths in continuous bulk ingestion (#121)
Mandar Inamdar [Mon, 2 Dec 2019 14:59:22 +0000 (07:59 -0700)] 
Fix #120 Support non default/absolute hdfs paths in continuous bulk ingestion (#121)

* Accept non default/absolute hdfs paths too as a parameter to cingest bulk
* Without this change, only defaultFS works
* Without this change, it gives error for any parameter other than relative path
* This feature support is critical, especially in case of multiple hdfs volumes
* Consistent usage of fs.makeQualified and reusing existing bulkDir variable

2 years agoFix #37 Restart GC during agitation (#117)
billierinaldi [Tue, 12 Nov 2019 21:52:39 +0000 (13:52 -0800)] 
Fix #37 Restart GC during agitation (#117)

2 years agoMove configuration of checkstyle plugin
Mike Miller [Thu, 7 Nov 2019 20:15:14 +0000 (15:15 -0500)] 
Move configuration of checkstyle plugin

2 years agoRemove explicit checkstyle dependency
Mike Miller [Wed, 6 Nov 2019 22:59:16 +0000 (17:59 -0500)] 
Remove explicit checkstyle dependency

2 years agoSupport extra Java options (#116)
Arvind Shyamsundar [Mon, 28 Oct 2019 22:32:32 +0000 (15:32 -0700)] 
Support extra Java options (#116)

Interjects an extra $JAVA_OPTS environemnt variable wherever Java is
being launched. This adds flexibility for the user to define extra
options, for example Xmx.

2 years agoFixes issue #114, incorrect auths used by scanners and walkers (#115)
Mandar Inamdar [Mon, 7 Oct 2019 23:44:11 +0000 (16:44 -0700)] 
Fixes issue #114, incorrect auths used by scanners and walkers (#115)

Fixed authValue.split("|") call in ContinuousEnv.java.
It was not considering | as a metacharacter in regex used by split
added escape character \\ in the split function call as authValue.split("\\|")

2 years agoREADME.md for automateEC2.sh (#113)
Russell Carter [Thu, 26 Sep 2019 20:05:31 +0000 (16:05 -0400)] 
README.md for automateEC2.sh  (#113)

2 years agoResolves #78 Script to automate EC2 (#96)
Russell Carter [Fri, 20 Sep 2019 16:09:52 +0000 (12:09 -0400)] 
Resolves #78 Script to automate EC2  (#96)

2 years agoUpdate ContinuousInputFormat to use col vis
Mike Miller [Thu, 19 Sep 2019 22:33:26 +0000 (18:33 -0400)] 
Update ContinuousInputFormat to use col vis

2 years agoUpdate cluster-control.sh.uno with uno updates
Mike Miller [Tue, 17 Sep 2019 20:52:52 +0000 (16:52 -0400)] 
Update cluster-control.sh.uno with uno updates

2 years agoAdd check for Test class to peformance script
Mike Miller [Tue, 17 Sep 2019 20:23:30 +0000 (16:23 -0400)] 
Add check for Test class to peformance script

2 years agoFixes #59 Create herding performance test (#93)
Laura Schanno [Tue, 17 Sep 2019 19:26:10 +0000 (15:26 -0400)] 
Fixes #59 Create herding performance test (#93)

* Add HeardingPT.java in order to test herding performance.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.

Related: #53

2 years agoMove performance tests to accumulo-testing #1200 (#90)
Laura Schanno [Tue, 17 Sep 2019 15:45:25 +0000 (11:45 -0400)] 
Move performance tests to accumulo-testing #1200 (#90)

* Move ManySplitIT as HighSplitCreationPT.
* Move BalanceFasterIT as SplitBalancingPT.
* Move DeleteTableDuringSplitIT as TableDeletionDuringSplitPT.
* Move RollWALPerformanceIT as RollWALPT.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.

Related: #1200

2 years agoFix BulkImport concurrent rwalk
Mike Miller [Thu, 12 Sep 2019 17:06:55 +0000 (13:06 -0400)] 
Fix BulkImport concurrent rwalk

2 years agoFix error in Dockerfile. Closes #109
Mike Miller [Wed, 11 Sep 2019 19:49:09 +0000 (15:49 -0400)] 
Fix error in Dockerfile. Closes #109

2 years agoDrop example files (#100)
Mike Miller [Thu, 15 Aug 2019 21:51:29 +0000 (17:51 -0400)] 
Drop example files (#100)

* Drop .example files
* Remove references to these files from documentation

2 years agoUse Accumulo 2.0.0 in testing repo
Christopher Tubbs [Sun, 4 Aug 2019 19:51:11 +0000 (15:51 -0400)] 
Use Accumulo 2.0.0 in testing repo

2 years agoFix warnings and clean up pom.xml
Christopher Tubbs [Tue, 30 Jul 2019 21:19:01 +0000 (17:19 -0400)] 
Fix warnings and clean up pom.xml

POM updates:
* Update plugins
* Add m2e profile for ensuring compiler compliance and ignoring plugins
  in Eclipse that Eclipse doesn't understand
* Require minimum java version of 1.8 and Maven 3.5
* Remove unneeded plugins (felix bundle and exec-maven-plugin)

Fix warnings in code:
* Use try-with-resources for TestEnv
* Use Double.parseDouble instead of new Double
* Compute iterable sizes from scanners without unused variables
* Suppress deprecation warnings where needed
* Update BulkPlusOne RW test to use new bulk import API
* Add missing case statement for GET_SUMMARIES in TableOp
* Remove unused variables

2 years agoRevert to Java 8 for accumulo-testing
Christopher Tubbs [Tue, 30 Jul 2019 20:25:46 +0000 (16:25 -0400)] 
Revert to Java 8 for accumulo-testing

Allow accumulo-testing to build and run with Java 8, but support the
release flag on JDK9 and later.

2 years agoUse non blocking secure random for seeding. (#106)
Keith Turner [Tue, 30 Jul 2019 18:17:24 +0000 (14:17 -0400)] 
Use non blocking secure random for seeding. (#106)

When attempting to generate bulk data to ingest, map reduce jobs were
timing out.  The jobs were blocked in SecureRandom code for over 10
minutes (likely trying to read /dev/random, but not sure about this).
This change creates a SecureRandom that should be non-blocking
with Java's default configs. With this change generating bulk data no
longer times out.

2 years agoUpdate accumulo-testing to use Java 11
Christopher Tubbs [Tue, 30 Jul 2019 00:18:07 +0000 (20:18 -0400)] 
Update accumulo-testing to use Java 11

2 years agoClean up and Minor Changes to ConditionalMutationsPT (#105)
Jeffrey Manno [Fri, 26 Jul 2019 18:55:22 +0000 (14:55 -0400)] 
Clean up and Minor Changes to ConditionalMutationsPT (#105)

2 years agoAvoid exceeding maximum log length in Travis
Christopher Tubbs [Thu, 25 Jul 2019 00:10:33 +0000 (20:10 -0400)] 
Avoid exceeding maximum log length in Travis

Suppress Downloading messages from Maven to avoid exceeding Travis CI's
maximum log length of 4MB.

2 years agoUse openjdk8 in Travis (oraclejdk8 gone)
Christopher Tubbs [Wed, 24 Jul 2019 23:52:14 +0000 (19:52 -0400)] 
Use openjdk8 in Travis (oraclejdk8 gone)

2 years agoImprove bulk CI docs (#104)
Keith Turner [Wed, 24 Jul 2019 23:50:06 +0000 (19:50 -0400)] 
Improve bulk CI docs (#104)

2 years agoReplace Thrift types with proper API Exception (#99)
Mike Miller [Wed, 24 Jul 2019 17:00:17 +0000 (13:00 -0400)] 
Replace Thrift types with proper API Exception (#99)

2 years agoFix #30 Test performance of Conditional Mutations (#79)
elinaawise [Wed, 24 Jul 2019 00:33:21 +0000 (20:33 -0400)] 
Fix #30 Test performance of Conditional Mutations (#79)

2 years agoUpdate documented expected CI bulk ingest counts (#103)
Keith Turner [Tue, 23 Jul 2019 16:14:17 +0000 (12:14 -0400)] 
Update documented expected CI bulk ingest counts (#103)

After the changes in #101 the counts produced by CI bulk ingest are more predictable.

2 years agoBulkIngest CI improvements (#101)
Christopher Tubbs [Tue, 23 Jul 2019 16:12:34 +0000 (12:12 -0400)] 
BulkIngest CI improvements (#101)

* Add job name
* Seed Random with strong seed
* Fix linked list to chain adjacent elements, rather than skip over
  adjacent elements to link to the next element

2 years agoFixes from shellcheck. Closes #69 (#102)
Mike Miller [Tue, 23 Jul 2019 15:15:33 +0000 (11:15 -0400)] 
Fixes from shellcheck. Closes #69 (#102)

2 years agoImprove scripts (#91)
Mike Miller [Mon, 22 Jul 2019 20:05:34 +0000 (16:05 -0400)] 
Improve scripts (#91)

* Made scripts all call ./bin/build
* Added check for env vars in env.sh
* Print out where the env vars are coming from

2 years agoUpdate ci bulk ingest docs (#98)
Keith Turner [Fri, 19 Jul 2019 00:37:52 +0000 (20:37 -0400)] 
Update ci bulk ingest docs (#98)

2 years agoFix #88 incorrect argument array indexes (#95)
Laura Schanno [Tue, 16 Jul 2019 22:03:29 +0000 (18:03 -0400)] 
Fix #88 incorrect argument array indexes (#95)

Update reference to correct position in array for the table name.

3 years agoupdate bulk docs (#85)
Keith Turner [Mon, 1 Jul 2019 18:42:38 +0000 (14:42 -0400)] 
update bulk docs (#85)

3 years agoFix shellcheck issues in conf dir
Christopher Tubbs [Wed, 26 Jun 2019 20:48:32 +0000 (16:48 -0400)] 
Fix shellcheck issues in conf dir

3 years agoFix #83 Add hadoop conf to CLASSPATH for defaultFS (#87)
Christopher Tubbs [Wed, 26 Jun 2019 20:47:22 +0000 (16:47 -0400)] 
Fix #83 Add hadoop conf to CLASSPATH for defaultFS (#87)

Ensure test client code has core-site.xml on its class path so that when
client code calls "new Configuration()" (for example, to qualify HDFS
paths without a scheme), it will use the fs.defaultFS to qualify them.

This fixes issues in bulk import test code that does not properly
qualify Paths as hdfs:// paths, and instead assumes file:// incorrectly.

3 years agoFix split point code for bulk ingest
Christopher Tubbs [Tue, 25 Jun 2019 20:32:05 +0000 (16:32 -0400)] 
Fix split point code for bulk ingest

3 years agoAdd type safety to State (#86)
Mike Miller [Tue, 25 Jun 2019 19:30:00 +0000 (15:30 -0400)] 
Add type safety to State (#86)

* Clean up code through reuse of new type safe State methods in bulk and concurrent modules
* Allowed removal of a bunch of warning suppresses

3 years agoSome improvements to rwalk script (#82)
Mike Miller [Thu, 20 Jun 2019 17:42:47 +0000 (13:42 -0400)] 
Some improvements to rwalk script (#82)

* Some improvements to ./bin/rwalk
* Fix script name

3 years agoadd gcs to docker
Keith Turner [Wed, 19 Jun 2019 20:06:03 +0000 (16:06 -0400)] 
add gcs to docker

3 years agoadd gcs to test list in readme
Keith Turner [Tue, 18 Jun 2019 20:07:25 +0000 (16:07 -0400)] 
add gcs to test list in readme

3 years agoAdd new garbage collection simulation test (#81)
Keith Turner [Tue, 18 Jun 2019 18:59:39 +0000 (14:59 -0400)] 
Add new garbage collection simulation test (#81)

3 years agoUpdate checkstyle (#80)
Mike Miller [Wed, 12 Jun 2019 21:08:27 +0000 (17:08 -0400)] 
Update checkstyle (#80)

* Fixes CVE-2019-9658

3 years agoRemove Snappy Property (#77)
Jeffrey Manno [Mon, 3 Jun 2019 16:55:14 +0000 (12:55 -0400)] 
Remove Snappy Property (#77)

3 years agoUpdated Hadoop and Zookeeper versions in the pom file (#76)
Jeffrey Zeiberg [Thu, 30 May 2019 17:55:41 +0000 (13:55 -0400)] 
Updated Hadoop and Zookeeper versions in the pom file (#76)

3 years agoAdd Durability Performance Test to Fix Durability IT (#74)
Jeffrey Manno [Tue, 21 May 2019 18:58:00 +0000 (14:58 -0400)] 
Add Durability Performance Test to Fix Durability IT (#74)

3 years agoUse Uno in upgrade test script and update for 2.0 (#73)
Keith Turner [Mon, 6 May 2019 18:40:05 +0000 (14:40 -0400)] 
Use Uno in upgrade test script and update for 2.0 (#73)

3 years agoCleanup Continuous Ingest (#71)
Mike Miller [Thu, 28 Mar 2019 19:06:18 +0000 (15:06 -0400)] 
Cleanup Continuous Ingest (#71)

* Add SLF4J logging
* Some minor cleanup

3 years agoReplace C-style arrays (#70)
Mike Miller [Wed, 27 Mar 2019 20:35:22 +0000 (16:35 -0400)] 
Replace C-style arrays (#70)

3 years agoRefactored RowHash and TeraSortIngest (#68)
Mike Walch [Thu, 21 Mar 2019 20:05:39 +0000 (16:05 -0400)] 
Refactored RowHash and TeraSortIngest (#68)

3 years agoUpdate Accumulo version to 2.0.0-SNAPSHOT (#66)
Mike Walch [Mon, 18 Mar 2019 14:55:32 +0000 (10:55 -0400)] 
Update Accumulo version to 2.0.0-SNAPSHOT (#66)

3 years agoUpdate ClientOpts (#65)
Mike Walch [Thu, 14 Mar 2019 22:47:35 +0000 (18:47 -0400)] 
Update ClientOpts (#65)

* Reduce the number of options
* Create clients using Accumulo.newClient()
* Remove use MapReduce opts from Accumulo

3 years agoMade CI bulk import configurable and scriptable (#64)
Keith Turner [Fri, 8 Mar 2019 23:03:07 +0000 (18:03 -0500)] 
Made CI bulk import configurable and scriptable (#64)

3 years agoApply formatter changes on build
Christopher Tubbs [Thu, 7 Mar 2019 23:03:40 +0000 (18:03 -0500)] 
Apply formatter changes on build

3 years agoUpdate pom.xml
Christopher Tubbs [Thu, 7 Mar 2019 22:59:14 +0000 (17:59 -0500)] 
Update pom.xml

Update apache parent pom and update formatting plugins

3 years agoCreates continuous BulkIngest (#63)
Mike Miller [Thu, 7 Mar 2019 22:54:51 +0000 (17:54 -0500)] 
Creates continuous BulkIngest (#63)

* A new Bulk import MapReduce job run using "cingest bulk"
* Will create a million random keys in Rfiles in the same format as CI
* The keys are in a linked list with the value being the prev row
* Works with CI createTable and verify

3 years agoClean up CLI parsing and stop depending on Accumulo internals (#61)
Mike Walch [Wed, 27 Feb 2019 20:07:29 +0000 (15:07 -0500)] 
Clean up CLI parsing and stop depending on Accumulo internals (#61)

* Remove dependencies on Accumulo internal CLI parsing code
  by addeing ClientOpts and Help
* Remove trace code from TestIngest

3 years agoFactor RFile and CryptoServiceFactory from Import-Control (#60)
Jeffrey Manno [Wed, 27 Feb 2019 17:21:59 +0000 (12:21 -0500)] 
Factor RFile and CryptoServiceFactory from Import-Control (#60)

3 years agoUpdate CI compaction strat settings (#58)
Keith Turner [Fri, 22 Feb 2019 18:44:27 +0000 (13:44 -0500)] 
Update CI compaction strat settings (#58)

Update the continuous ingest table settings to reflect the changes made
in apache/accumulo#935

3 years agoImprove yield+scan exec performance test (#57)
Keith Turner [Fri, 22 Feb 2019 18:43:05 +0000 (13:43 -0500)] 
Improve yield+scan exec performance test (#57)

Before this commit the filters running in the background had a very low
chance of returning data.  This did not execercise the use case of an
iterator returning data very slowly.  Returning data very slowly results
in slowly filling a buffer.  The test was modified to run the filter
scans with multiple probabilites which causes data to return with
different velocities.  Also the yield filter was modfied to support
yielding across returning a key/value.

The design of the yield filter was improved so that subclasses do not
have to override init().

3 years agoUpdate japi readme and master config (#55)
Mike Miller [Fri, 8 Feb 2019 20:21:24 +0000 (15:21 -0500)] 
Update japi readme and master config (#55)

* Update japi readme and master config

* Update japi-accumulo-master.xml

Remove old jar

3 years agoAdded perf test for yielding+scan dispatching (#56)
Keith Turner [Fri, 8 Feb 2019 20:05:06 +0000 (15:05 -0500)] 
Added perf test for yielding+scan dispatching (#56)

Test running lots of short scans while many filters that return little
data are running in the background.  If nothing is done these filters
will prevent the short scans from running.  This test configures
Accumulo so that the short scans should get a chance to run.  It does
three things to facilitate the short scans : the filters yield, there
are two scan executors, and a scan dispatcher sends long running scans
to the second executor. If yielding and dispatching are working
correctly then the short scans should have very short response times.
This happens because the filters should end up in a separate thread pool
than the short scan.

Also fixed some bugs with the performance test framework.

3 years agoUpdate japi-accumulo-master.xml 55/head
Mike Miller [Fri, 8 Feb 2019 17:48:32 +0000 (12:48 -0500)] 
Update japi-accumulo-master.xml

Remove old jar

3 years agoUpdate japi readme and master config
Mike Miller [Thu, 7 Feb 2019 23:23:38 +0000 (18:23 -0500)] 
Update japi readme and master config

3 years agoAllowed configuring table props for CI table (#54)
Keith Turner [Thu, 31 Jan 2019 18:41:58 +0000 (13:41 -0500)] 
Allowed configuring table props for CI table (#54)

3 years agoStopped using non public API ClientContext
Keith Turner [Wed, 30 Jan 2019 17:31:07 +0000 (12:31 -0500)] 
Stopped using non public API ClientContext

Needed to do this inorder to build against 2.0.0-alpha-2

3 years agoUpdate checkstyle to use import-control for API check (#52)
Mike Miller [Tue, 29 Jan 2019 19:09:19 +0000 (14:09 -0500)] 
Update checkstyle to use import-control for API check (#52)

* Update checkstyle to use import-control for checking use of accumulo API
* Add checkstyle plugin to the build execution to always run
* Add exceptions to API checking to hold the line until we can fix
* Use Guava peeking iterator in place of accumulo peeking iterator

3 years agoRemove CachedConfiguration to fix build
Mike Miller [Mon, 28 Jan 2019 16:05:44 +0000 (11:05 -0500)] 
Remove CachedConfiguration to fix build

3 years agoImprovements made for 2.0.0-alpha-2 CI testing (#51)
Keith Turner [Sat, 26 Jan 2019 22:03:41 +0000 (17:03 -0500)] 
Improvements made for 2.0.0-alpha-2 CI testing (#51)

* Improvements made for 2.0.0-alpha-2 CI testing

Added ability to set test options on command line. This makes running different
types of ingest from the same docker image easy.  Without this change a new
docker image would need to be created for each different ingest type.

Documented commands for running a long running ingest test.

Made the ingest flush size configurable.  This made it possible to write really
small amounts of data and then pause.  This changes makes testing
for the problem in apache/accumulo#854 possible.

Added docs for how to copy docker image to other nodes if you don't have
a repository handy.

3 years agoAdd commons codec depdency to pom
Keith Turner [Wed, 23 Jan 2019 23:17:49 +0000 (18:17 -0500)] 
Add commons codec depdency to pom

Tried building against alpha-2 release and it failed to find commons
codec.  Maybe it used to get it transitively from Accumulo.

3 years agoUpdates for starting MapReduce jobs from randomwalk (#49)
Mike Walch [Thu, 10 Jan 2019 17:38:56 +0000 (12:38 -0500)] 
Updates for starting MapReduce jobs from randomwalk (#49)

* Set additional MapReduce configuration and Hadoop username
* These settings work if running randomwalk on user machine
  or in Docker

3 years agoUpdate copyright date to 2019
Mike Walch [Wed, 2 Jan 2019 18:47:15 +0000 (13:47 -0500)] 
Update copyright date to 2019

3 years agoUpdates due to Accumulo 2.0 changes (#48)
Mike Walch [Wed, 2 Jan 2019 17:50:52 +0000 (12:50 -0500)] 
Updates due to Accumulo 2.0 changes (#48)

* Use new MapReduce API and client classloader
* Use createClient and close client
* Updated codestyle.xml to match file in Accumulo repo