Jeffrey Manno [Tue, 19 Oct 2021 17:24:38 +0000 (13:24 -0400)]
ignore replication IT inside accumulo-testing
Dom G [Tue, 5 Oct 2021 14:30:11 +0000 (10:30 -0400)]
Fix failing ScanExecutorPT (#163)
* replaced outdated hinting
Co-authored-by: Keith Turner <kturner@apache.org>
Co-authored-by: Jeffrey Manno <jeffreymanno15@gmail.com>
Jeffrey Manno [Wed, 29 Sep 2021 13:08:20 +0000 (09:08 -0400)]
Update readme and add comments to RandomCachedLookupsPT
* These additions aim to inform the user that some tests may require changes to certain property values in order to run.
Jeffrey Manno [Tue, 28 Sep 2021 21:26:40 +0000 (17:26 -0400)]
Add documentation, comments, and warning fixes
Dom G [Fri, 24 Sep 2021 15:23:03 +0000 (11:23 -0400)]
Add units to performance test results (#156)
* Add units to PT output
* Added constants and corrections
* Updated results descriptions
Dom G [Thu, 23 Sep 2021 20:25:18 +0000 (16:25 -0400)]
Fix performance test NumberFormatException bug (#155)
* Fixed NumberFormatException
* Add constants, make runs consistent
* Make var names more meaningful
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
Dom G [Thu, 23 Sep 2021 14:04:54 +0000 (10:04 -0400)]
Add example of a single performance test run (#158)
* Update readme to include single performance test run example
* Misc. small grammar fixes
AlbertWhitlock [Mon, 20 Sep 2021 18:12:12 +0000 (14:12 -0400)]
Migrate accumulo-testing to log4j2. (#146)
Migrate accumulo-testing to use log4j2 (via slf4j)
Co-authored-by: Jeffrey Manno <jeffreymanno15@gmail.com>
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
Dom G [Fri, 17 Sep 2021 16:01:09 +0000 (12:01 -0400)]
Create unique directory for each run of `cingest verify` to store test output (#152)
* Now creates a new unique directory to store test output of each run of `cingest verify
* Now no longer stops the test when the output directory already exists
Dom G [Fri, 10 Sep 2021 19:08:52 +0000 (15:08 -0400)]
replace use of deprecated newInstance() (#151)
Replaced the use of the deprecated newInstance() method with its replacement getDeclaredConstructor().newInstance()
Dom G [Fri, 10 Sep 2021 19:06:59 +0000 (15:06 -0400)]
Refactor out Non API code (#150)
Removed or replaced usages of non-API code
Dom G [Thu, 2 Sep 2021 12:25:00 +0000 (08:25 -0400)]
Merge pull request #149 from DomGarguilo/updateCI
* Add capability to change compaction planner/dispatcher
* Refactored prop name
* Remove external comps
domgarguilo [Mon, 23 Aug 2021 15:07:11 +0000 (11:07 -0400)]
Remove external comps. and replace foreach
domgarguilo [Tue, 17 Aug 2021 17:53:07 +0000 (13:53 -0400)]
Refactored prop name
domgarguilo [Tue, 17 Aug 2021 17:44:04 +0000 (13:44 -0400)]
Add capability to change compaction planner/dispatcher
EdColeman [Mon, 16 Aug 2021 21:48:00 +0000 (17:48 -0400)]
Merge pull request #143 from apache/update_guava_version_30_1_1
Update guava version to 30.1.1 (matches accumulo project)
Ed Coleman etcoleman [Mon, 16 Aug 2021 21:31:02 +0000 (21:31 +0000)]
Update guava version (rebased to current main)
Jeffrey Manno [Mon, 9 Aug 2021 16:06:26 +0000 (12:06 -0400)]
Replace deprecated prop inside ReplicationRandomWalkIT with its replacement
Jeffrey Manno [Mon, 9 Aug 2021 15:42:43 +0000 (11:42 -0400)]
Replace deprecated prop inside ReplicationRandomWalkIT
AlbertWhitlock [Tue, 1 Jun 2021 14:39:09 +0000 (10:39 -0400)]
Properly build shaded jar when running cingest (#140)
Mike Miller [Wed, 14 Apr 2021 11:53:38 +0000 (07:53 -0400)]
Update pom to Java 11 (#138)
* Update pom to Java 11
* Remove obsoleted profile
* Use java 11 for m2e profile as well
Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
dependabot[bot] [Thu, 1 Apr 2021 21:42:39 +0000 (17:42 -0400)]
Bump guava from 26.0-jre to 29.0-jre (#137)
Bumps [guava](https://github.com/google/guava) from 26.0-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Mike Miller [Tue, 23 Feb 2021 13:28:57 +0000 (08:28 -0500)]
Fix typo in bin/cingest script
Mike Miller [Mon, 22 Feb 2021 19:40:12 +0000 (14:40 -0500)]
Add back END to MultiTable so it can be used by other modules
Mike Miller [Wed, 17 Feb 2021 14:21:10 +0000 (09:21 -0500)]
Add more splits to RW multitable CreateTable
Jeffrey Manno [Thu, 11 Feb 2021 18:40:30 +0000 (13:40 -0500)]
Fixes #135. Fix build errors due recent changes in accumulo and upgrade accumulo version to 2.1.0
Jeffrey Manno [Thu, 11 Feb 2021 14:27:39 +0000 (09:27 -0500)]
upgrade accumulo to 2.1.0-SNAPSHOT
Jeffrey Manno [Thu, 11 Feb 2021 14:24:58 +0000 (09:24 -0500)]
add import class exception for ThreadPools
Jeffrey Manno [Thu, 11 Feb 2021 13:37:49 +0000 (08:37 -0500)]
fix build errors
Mike Miller [Tue, 9 Feb 2021 21:48:55 +0000 (16:48 -0500)]
Add a Bulk import to randomwalk MultiTable (#134)
* Create new BulkImport test in MultiTable for more realistic case
* Tweak MultiTable.xml to drop 20 tables, same as create. This will keep
the number of talbes down when running for long period.
* Also set log4j testing logging to DEBUG
Mike Miller [Fri, 8 Jan 2021 13:15:28 +0000 (08:15 -0500)]
Exclude log4j2 jars from shade plugin (#133)
Mike Miller [Tue, 5 Jan 2021 12:48:13 +0000 (07:48 -0500)]
Fix warning in randomwalk Init
* Fix redundant cast to java.util.Random warning in randomwalk Init
rchoksi-hw [Mon, 14 Dec 2020 15:09:04 +0000 (07:09 -0800)]
update test with correct API call (#131)
Co-authored-by: romil choksi <rchoksi@cloudera.com>
Christopher Tubbs [Tue, 13 Oct 2020 02:04:15 +0000 (22:04 -0400)]
Upload test logs on build failure
dependabot[bot] [Tue, 13 Oct 2020 01:13:43 +0000 (21:13 -0400)]
Bump junit from 4.12 to 4.13.1 (#129)
Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Christopher Tubbs [Mon, 5 Oct 2020 21:16:30 +0000 (17:16 -0400)]
Add .asf.yaml (#128)
Keith Turner [Mon, 5 Oct 2020 16:54:45 +0000 (12:54 -0400)]
Merge pull request #125 from keith-turner/gcpt
Add group commit performance test
Keith Turner [Mon, 28 Sep 2020 17:59:53 +0000 (13:59 -0400)]
removes uneeded import from performance script (#126)
Keith Turner [Sat, 26 Sep 2020 01:18:33 +0000 (21:18 -0400)]
presplit table
Keith Turner [Fri, 25 Sep 2020 19:14:33 +0000 (15:14 -0400)]
fixes #67 improves group commit performance test
Elina Wise [Fri, 24 May 2019 16:25:11 +0000 (12:25 -0400)]
Fix #67: Test group commit performance
Mark Owens [Tue, 22 Sep 2020 20:02:08 +0000 (16:02 -0400)]
Missing license headers #122 (#124)
* Added missing license headers to 20+ files.
* Added the apache-rat plugin to the pom file.
* Moved the splits data files in test/bench/lib into a data
subdirectory.
* Modified TableSplitsBenchmark.py to account for the movement of the
splits file into the data directory.
* Added exclusions for the data directory and the exlude_classes.txt
file.
* Update .gitignore to ignore .pyc files.
Christopher Tubbs [Tue, 11 Aug 2020 13:27:41 +0000 (09:27 -0400)]
Update for main branch and GitHub Actions CI
Mark Owens [Wed, 27 May 2020 16:41:12 +0000 (12:41 -0400)]
Update README.md
While reading over the accumulo-testing readme, I noticed a couple of typos. This update corrects the ones I found.
Tushar D [Thu, 14 Nov 2019 02:17:32 +0000 (18:17 -0800)]
Adding Accumulo Availability Monitor :
Monitor.Java : This file includes core code for scan operation that randomly picks a tablet server, identifies its minimum and maximum rows range and picks out 'distance' (configurable through opts) number of rows to be scanned in one iteration.
MonitorOpts.java : This file is used to provide all the parameters which are taken by the Monitor Class. Interesting Options Include : Ability to set scanner sleep time , between iterations. Setting Batch Size that scanner uses internally. Setting number of iteration or continuous mode.
monitor script : This follows same pattern as cingest, rwalk, performance scripts.
Caching Splits
Schema Agnostic Healthprobe
Added Changes to make the healthprobe schema agnostic, while retaining the functionality to scan configurable number of rows. Rows are scanned from beginning of the tablet. Tablet information is loaded once and cached for later iterations.
Signed-off-by: Marc Parisi <phrocker@apache.org>
Mandar Inamdar [Mon, 2 Dec 2019 14:59:22 +0000 (07:59 -0700)]
Fix #120 Support non default/absolute hdfs paths in continuous bulk ingestion (#121)
* Accept non default/absolute hdfs paths too as a parameter to cingest bulk
* Without this change, only defaultFS works
* Without this change, it gives error for any parameter other than relative path
* This feature support is critical, especially in case of multiple hdfs volumes
* Consistent usage of fs.makeQualified and reusing existing bulkDir variable
billierinaldi [Tue, 12 Nov 2019 21:52:39 +0000 (13:52 -0800)]
Fix #37 Restart GC during agitation (#117)
Mike Miller [Thu, 7 Nov 2019 20:15:14 +0000 (15:15 -0500)]
Move configuration of checkstyle plugin
Mike Miller [Wed, 6 Nov 2019 22:59:16 +0000 (17:59 -0500)]
Remove explicit checkstyle dependency
Arvind Shyamsundar [Mon, 28 Oct 2019 22:32:32 +0000 (15:32 -0700)]
Support extra Java options (#116)
Interjects an extra $JAVA_OPTS environemnt variable wherever Java is
being launched. This adds flexibility for the user to define extra
options, for example Xmx.
Mandar Inamdar [Mon, 7 Oct 2019 23:44:11 +0000 (16:44 -0700)]
Fixes issue #114, incorrect auths used by scanners and walkers (#115)
Fixed authValue.split("|") call in ContinuousEnv.java.
It was not considering | as a metacharacter in regex used by split
added escape character \\ in the split function call as authValue.split("\\|")
Russell Carter [Thu, 26 Sep 2019 20:05:31 +0000 (16:05 -0400)]
README.md for automateEC2.sh (#113)
Russell Carter [Fri, 20 Sep 2019 16:09:52 +0000 (12:09 -0400)]
Resolves #78 Script to automate EC2 (#96)
Mike Miller [Thu, 19 Sep 2019 22:33:26 +0000 (18:33 -0400)]
Update ContinuousInputFormat to use col vis
Mike Miller [Tue, 17 Sep 2019 20:52:52 +0000 (16:52 -0400)]
Update cluster-control.sh.uno with uno updates
Mike Miller [Tue, 17 Sep 2019 20:23:30 +0000 (16:23 -0400)]
Add check for Test class to peformance script
Laura Schanno [Tue, 17 Sep 2019 19:26:10 +0000 (15:26 -0400)]
Fixes #59 Create herding performance test (#93)
* Add HeardingPT.java in order to test herding performance.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.
Related: #53
Laura Schanno [Tue, 17 Sep 2019 15:45:25 +0000 (11:45 -0400)]
Move performance tests to accumulo-testing #1200 (#90)
* Move ManySplitIT as HighSplitCreationPT.
* Move BalanceFasterIT as SplitBalancingPT.
* Move DeleteTableDuringSplitIT as TableDeletionDuringSplitPT.
* Move RollWALPerformanceIT as RollWALPT.
* Use Result.result() instead of Result.parameter() when reporting
resulting values so that they will be put in the appropriate section
when the JSON files are written.
Related: #1200
Mike Miller [Thu, 12 Sep 2019 17:06:55 +0000 (13:06 -0400)]
Fix BulkImport concurrent rwalk
Mike Miller [Wed, 11 Sep 2019 19:49:09 +0000 (15:49 -0400)]
Fix error in Dockerfile. Closes #109
Mike Miller [Thu, 15 Aug 2019 21:51:29 +0000 (17:51 -0400)]
Drop example files (#100)
* Drop .example files
* Remove references to these files from documentation
Christopher Tubbs [Sun, 4 Aug 2019 19:51:11 +0000 (15:51 -0400)]
Use Accumulo 2.0.0 in testing repo
Christopher Tubbs [Tue, 30 Jul 2019 21:19:01 +0000 (17:19 -0400)]
Fix warnings and clean up pom.xml
POM updates:
* Update plugins
* Add m2e profile for ensuring compiler compliance and ignoring plugins
in Eclipse that Eclipse doesn't understand
* Require minimum java version of 1.8 and Maven 3.5
* Remove unneeded plugins (felix bundle and exec-maven-plugin)
Fix warnings in code:
* Use try-with-resources for TestEnv
* Use Double.parseDouble instead of new Double
* Compute iterable sizes from scanners without unused variables
* Suppress deprecation warnings where needed
* Update BulkPlusOne RW test to use new bulk import API
* Add missing case statement for GET_SUMMARIES in TableOp
* Remove unused variables
Christopher Tubbs [Tue, 30 Jul 2019 20:25:46 +0000 (16:25 -0400)]
Revert to Java 8 for accumulo-testing
Allow accumulo-testing to build and run with Java 8, but support the
release flag on JDK9 and later.
Keith Turner [Tue, 30 Jul 2019 18:17:24 +0000 (14:17 -0400)]
Use non blocking secure random for seeding. (#106)
When attempting to generate bulk data to ingest, map reduce jobs were
timing out. The jobs were blocked in SecureRandom code for over 10
minutes (likely trying to read /dev/random, but not sure about this).
This change creates a SecureRandom that should be non-blocking
with Java's default configs. With this change generating bulk data no
longer times out.
Christopher Tubbs [Tue, 30 Jul 2019 00:18:07 +0000 (20:18 -0400)]
Update accumulo-testing to use Java 11
Jeffrey Manno [Fri, 26 Jul 2019 18:55:22 +0000 (14:55 -0400)]
Clean up and Minor Changes to ConditionalMutationsPT (#105)
Christopher Tubbs [Thu, 25 Jul 2019 00:10:33 +0000 (20:10 -0400)]
Avoid exceeding maximum log length in Travis
Suppress Downloading messages from Maven to avoid exceeding Travis CI's
maximum log length of 4MB.
Christopher Tubbs [Wed, 24 Jul 2019 23:52:14 +0000 (19:52 -0400)]
Use openjdk8 in Travis (oraclejdk8 gone)
Keith Turner [Wed, 24 Jul 2019 23:50:06 +0000 (19:50 -0400)]
Improve bulk CI docs (#104)
Mike Miller [Wed, 24 Jul 2019 17:00:17 +0000 (13:00 -0400)]
Replace Thrift types with proper API Exception (#99)
elinaawise [Wed, 24 Jul 2019 00:33:21 +0000 (20:33 -0400)]
Fix #30 Test performance of Conditional Mutations (#79)
Keith Turner [Tue, 23 Jul 2019 16:14:17 +0000 (12:14 -0400)]
Update documented expected CI bulk ingest counts (#103)
After the changes in #101 the counts produced by CI bulk ingest are more predictable.
Christopher Tubbs [Tue, 23 Jul 2019 16:12:34 +0000 (12:12 -0400)]
BulkIngest CI improvements (#101)
* Add job name
* Seed Random with strong seed
* Fix linked list to chain adjacent elements, rather than skip over
adjacent elements to link to the next element
Mike Miller [Tue, 23 Jul 2019 15:15:33 +0000 (11:15 -0400)]
Fixes from shellcheck. Closes #69 (#102)
Mike Miller [Mon, 22 Jul 2019 20:05:34 +0000 (16:05 -0400)]
Improve scripts (#91)
* Made scripts all call ./bin/build
* Added check for env vars in env.sh
* Print out where the env vars are coming from
Keith Turner [Fri, 19 Jul 2019 00:37:52 +0000 (20:37 -0400)]
Update ci bulk ingest docs (#98)
Laura Schanno [Tue, 16 Jul 2019 22:03:29 +0000 (18:03 -0400)]
Fix #88 incorrect argument array indexes (#95)
Update reference to correct position in array for the table name.
Keith Turner [Mon, 1 Jul 2019 18:42:38 +0000 (14:42 -0400)]
update bulk docs (#85)
Christopher Tubbs [Wed, 26 Jun 2019 20:48:32 +0000 (16:48 -0400)]
Fix shellcheck issues in conf dir
Christopher Tubbs [Wed, 26 Jun 2019 20:47:22 +0000 (16:47 -0400)]
Fix #83 Add hadoop conf to CLASSPATH for defaultFS (#87)
Ensure test client code has core-site.xml on its class path so that when
client code calls "new Configuration()" (for example, to qualify HDFS
paths without a scheme), it will use the fs.defaultFS to qualify them.
This fixes issues in bulk import test code that does not properly
qualify Paths as hdfs:// paths, and instead assumes file:// incorrectly.
Christopher Tubbs [Tue, 25 Jun 2019 20:32:05 +0000 (16:32 -0400)]
Fix split point code for bulk ingest
Mike Miller [Tue, 25 Jun 2019 19:30:00 +0000 (15:30 -0400)]
Add type safety to State (#86)
* Clean up code through reuse of new type safe State methods in bulk and concurrent modules
* Allowed removal of a bunch of warning suppresses
Mike Miller [Thu, 20 Jun 2019 17:42:47 +0000 (13:42 -0400)]
Some improvements to rwalk script (#82)
* Some improvements to ./bin/rwalk
* Fix script name
Keith Turner [Wed, 19 Jun 2019 20:06:03 +0000 (16:06 -0400)]
add gcs to docker
Keith Turner [Tue, 18 Jun 2019 20:07:25 +0000 (16:07 -0400)]
add gcs to test list in readme
Keith Turner [Tue, 18 Jun 2019 18:59:39 +0000 (14:59 -0400)]
Add new garbage collection simulation test (#81)
Mike Miller [Wed, 12 Jun 2019 21:08:27 +0000 (17:08 -0400)]
Update checkstyle (#80)
* Fixes CVE-2019-9658
Jeffrey Manno [Mon, 3 Jun 2019 16:55:14 +0000 (12:55 -0400)]
Remove Snappy Property (#77)
Jeffrey Zeiberg [Thu, 30 May 2019 17:55:41 +0000 (13:55 -0400)]
Updated Hadoop and Zookeeper versions in the pom file (#76)
Jeffrey Manno [Tue, 21 May 2019 18:58:00 +0000 (14:58 -0400)]
Add Durability Performance Test to Fix Durability IT (#74)
Keith Turner [Mon, 6 May 2019 18:40:05 +0000 (14:40 -0400)]
Use Uno in upgrade test script and update for 2.0 (#73)
Mike Miller [Thu, 28 Mar 2019 19:06:18 +0000 (15:06 -0400)]
Cleanup Continuous Ingest (#71)
* Add SLF4J logging
* Some minor cleanup
Mike Miller [Wed, 27 Mar 2019 20:35:22 +0000 (16:35 -0400)]
Replace C-style arrays (#70)
Mike Walch [Thu, 21 Mar 2019 20:05:39 +0000 (16:05 -0400)]
Refactored RowHash and TeraSortIngest (#68)
Mike Walch [Mon, 18 Mar 2019 14:55:32 +0000 (10:55 -0400)]
Update Accumulo version to 2.0.0-SNAPSHOT (#66)
Mike Walch [Thu, 14 Mar 2019 22:47:35 +0000 (18:47 -0400)]
Update ClientOpts (#65)
* Reduce the number of options
* Create clients using Accumulo.newClient()
* Remove use MapReduce opts from Accumulo
Keith Turner [Fri, 8 Mar 2019 23:03:07 +0000 (18:03 -0500)]
Made CI bulk import configurable and scriptable (#64)
Christopher Tubbs [Thu, 7 Mar 2019 23:03:40 +0000 (18:03 -0500)]
Apply formatter changes on build
Christopher Tubbs [Thu, 7 Mar 2019 22:59:14 +0000 (17:59 -0500)]
Update pom.xml
Update apache parent pom and update formatting plugins