drill-test-framework.git
5 weeks agoBump xercesImpl from 2.12.0 to 2.12.2 in /framework (#5) main
dependabot[bot] [Thu, 19 May 2022 00:03:31 +0000 (17:03 -0700)] 
Bump xercesImpl from 2.12.0 to 2.12.2 in /framework (#5)

6 weeks agoMerge pull request #1 from agozhiy/resurrection
James Turton [Tue, 10 May 2022 12:38:06 +0000 (14:38 +0200)] 
Merge pull request #1 from agozhiy/resurrection

Resurrection

7 weeks agoChanged logger implementation from log4j to slf4j w/ logback 1/head
Anton Gozhiy [Mon, 2 May 2022 18:58:54 +0000 (21:58 +0300)] 
Changed logger implementation from log4j to slf4j w/ logback

2 months agoRefactored remaining test suites to use data preparation phases instead of scripts resurrection
Anton Gozhiy [Thu, 14 Apr 2022 17:14:30 +0000 (20:14 +0300)] 
Refactored remaining test suites to use data preparation phases instead of scripts

Also updated README.md and CONTRIBUTING.md to reflect the changes

2 months agoAdded new data preparation phases
Anton Gozhiy [Sat, 12 Mar 2022 09:20:48 +0000 (11:20 +0200)] 
Added new data preparation phases

New preparation phases:
- "del" for deletion data that needs to be fresh
- "mkdir" for directory creation
- "ddl" for ddl files execution
- "post_cp" for copying previously generated data
Other changes:
- Added a new exclude option "all", so starting tests with "-x all" argument will not run test suites with any dependency
- Refactored test suites to use the newly added preparation phases instead of bash scripts
- Temporary excluded several suites where "gen" scripts were too complicated

4 months agoAdded docker maven plugin, changed defaults to make it build and run easier
Anton Gozhiy [Fri, 11 Feb 2022 17:52:09 +0000 (19:52 +0200)] 
Added docker maven plugin, changed defaults to make it build and run easier

23 months agoFixed links to the datasets on S3 (#608)
Anton Gozhiy [Tue, 30 Jun 2020 19:41:26 +0000 (22:41 +0300)] 
Fixed links to the datasets on S3 (#608)

2 years agoBump xercesImpl from 2.8.0 to 2.12.0 in /framework (#607)
dependabot[bot] [Mon, 15 Jun 2020 19:33:56 +0000 (12:33 -0700)] 
Bump xercesImpl from 2.8.0 to 2.12.0 in /framework (#607)

Bumps xercesImpl from 2.8.0 to 2.12.0.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years agoDRILL-7713: Baseline changes due to update of Metadata-extractor library (#606)
Anton Gozhiy [Wed, 22 Apr 2020 18:02:55 +0000 (21:02 +0300)] 
DRILL-7713: Baseline changes due to update of Metadata-extractor library (#606)

2 years agoDRILL-6168: Adjust results for table functions (#605)
Paul Rogers [Wed, 22 Apr 2020 14:51:19 +0000 (07:51 -0700)] 
DRILL-6168: Adjust results for table functions (#605)

Also edits to the README file

2 years agoBump zookeeper from 3.4.6 to 3.4.14 in /framework (#604)
dependabot[bot] [Mon, 6 Apr 2020 16:36:38 +0000 (09:36 -0700)] 
Bump zookeeper from 3.4.6 to 3.4.14 in /framework (#604)

Bumps zookeeper from 3.4.6 to 3.4.14.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 years agoDRILL-7615: Added tests to verify that decimal columns work correctly with UNION...
Dmytro Kondriukov [Mon, 6 Apr 2020 09:11:08 +0000 (12:11 +0300)] 
DRILL-7615: Added tests to verify that decimal columns work correctly with UNION operator (#603)

Also renamed existing tests according to the project naming convention

2 years agoDRILL-7502: fix baselines (#602)
Denys Ordynskiy [Thu, 9 Jan 2020 16:40:23 +0000 (18:40 +0200)] 
DRILL-7502: fix baselines (#602)

2 years agoDRILL-7409: Check if BigInt is read correctly with parquet in diction… (#601)
Denys Ordynskiy [Thu, 7 Nov 2019 23:25:28 +0000 (01:25 +0200)] 
DRILL-7409: Check if BigInt is read correctly with parquet in diction… (#601)

* DRILL-7409: Check if BigInt is read correctly with parquet in dictionary encoding.

* DRILL-7409: moved test data

2 years agoDRILL-7418: Fixed baselines due to optimizations (#600)
Anton Gozhiy [Wed, 30 Oct 2019 15:14:17 +0000 (17:14 +0200)] 
DRILL-7418: Fixed baselines due to optimizations (#600)

After DRILL-7418, we don't enumerate files used during direct count optimization, only selection root and number of files are listed.
For Hive selection root is unavailable thus only number of files will be present in plan.

2 years agoBump jackson.version from 2.9.9 to 2.10.0 in /framework (#599)
dependabot[bot] [Mon, 21 Oct 2019 19:04:10 +0000 (12:04 -0700)] 
Bump jackson.version from 2.9.9 to 2.10.0 in /framework (#599)

Bumps `jackson.version` from 2.9.9 to 2.10.0.

Updates `jackson-jaxrs-json-provider` from 2.9.9 to 2.10.0

Updates `jackson-core` from 2.9.9 to 2.10.0
- [Release notes](https://github.com/FasterXML/jackson-core/releases)
- [Commits](https://github.com/FasterXML/jackson-core/compare/jackson-core-2.9.9...jackson-core-2.10.0)

Updates `jackson-databind` from 2.9.9 to 2.10.0
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `jackson-annotations` from 2.9.9 to 2.10.0
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2 years agoSpecified query columns for a test query. (#598)
Denys Ordynskiy [Wed, 2 Oct 2019 16:57:06 +0000 (19:57 +0300)] 
Specified query columns for a test query. (#598)

2 years agoDRILL-7096: Fixed baselines due to added support for canonical Map (#597)
Anton Gozhiy [Tue, 3 Sep 2019 15:17:58 +0000 (18:17 +0300)] 
DRILL-7096: Fixed baselines due to added support for canonical Map (#597)

Changes in query output:
"map":[{"key":"<some_key>","value":"<some_value>"}] -> {"<some_key>":"<some_value>"}

2 years agoTest for parseError using WITH and UNION ALL operators (#593)
Pavel Semenov [Tue, 3 Sep 2019 15:00:56 +0000 (18:00 +0300)] 
Test for parseError using WITH and UNION ALL operators (#593)

* Test for parseError using WITH and UNION ALL operators

* Fix use statement in the test.

2 years agoTesting CRUD for the table schema. (#596)
Denys Ordynskiy [Fri, 23 Aug 2019 18:13:27 +0000 (21:13 +0300)] 
Testing CRUD for the table schema. (#596)

2 years agoAdded an option for float / double precision that would help to solve precision relat...
Anton Gozhiy [Tue, 13 Aug 2019 19:08:03 +0000 (22:08 +0300)] 
Added an option for float / double precision that would help to solve precision related test failures (#594)

2 years agoFixed baselines due to fixes in DRILL-4517 (#595)
Anton Gozhiy [Tue, 13 Aug 2019 19:06:59 +0000 (22:06 +0300)] 
Fixed baselines due to fixes in DRILL-4517 (#595)

Fixed cases with nulls in arrays, when Drill mistakenly considered some row groups as empty and removed them from scan.

2 years agoTest for CompileException in VarDecimalHolder class (#592)
Pavel Semenov [Wed, 31 Jul 2019 15:46:44 +0000 (18:46 +0300)] 
Test for CompileException in VarDecimalHolder class (#592)

2 years ago'Decimal' and 'prq_union_all' extention folder files refactoring (#591)
Pavel Semenov [Fri, 26 Jul 2019 13:04:07 +0000 (16:04 +0300)] 
'Decimal' and 'prq_union_all' extention folder files refactoring (#591)

2 years agoDRILL-7200: Baseline updates due to Calcite update (#590)
Anton Gozhiy [Fri, 5 Jul 2019 16:11:06 +0000 (19:11 +0300)] 
DRILL-7200: Baseline updates due to Calcite update (#590)

- After the Calcite update (1.20) a validation was added for duplicate aliases. Fixed by making them unique.
- Fixed plans where literals are now concatenated with type.

3 years agoAdded tests for V3 text reader issues fixes (#589)
Anton Gozhiy [Mon, 24 Jun 2019 21:35:26 +0000 (00:35 +0300)] 
Added tests for V3 text reader issues fixes (#589)

Tests:
DRILL_4814_table_function.sql
DRILL_5487_truncated_last_row.sql
DRILL_6952_enable_v3_text_reader_default_value.sql
DRILL_6952_implicit_columns.sql
DRILL_7034_window function.sql

Also:
- Changed tests extension to .sql.
- Aligned the code inside textReadGroup.json.

3 years agoDRILL-7268: Updated baselines for two Hive parquet files (#588)
Pavel Semenov [Mon, 24 Jun 2019 21:32:38 +0000 (00:32 +0300)] 
DRILL-7268: Updated baselines for two Hive parquet files (#588)

3 years agoDRILL-7292: Updated baselines due to transition to V3 Text Reader (#587)
Anton Gozhiy [Thu, 20 Jun 2019 06:59:26 +0000 (09:59 +0300)] 
DRILL-7292: Updated baselines due to transition to V3 Text Reader (#587)

Now all implicit columns displayed at the end.

3 years agoUpdated baselines due to changes in DRILL-4782 / DRILL-7139 (#583)
Anton Gozhiy [Fri, 31 May 2019 12:10:04 +0000 (15:10 +0300)] 
Updated baselines due to changes in DRILL-4782 / DRILL-7139 (#583)

3 years agoBump jackson.version from 2.8.11 to 2.9.9 in /framework (#586)
dependabot[bot] [Wed, 29 May 2019 18:21:53 +0000 (11:21 -0700)] 
Bump jackson.version from 2.8.11 to 2.9.9 in /framework (#586)

Bumps `jackson.version` from 2.8.11 to 2.9.9.

Updates `jackson-jaxrs-json-provider` from 2.8.11 to 2.9.9

Updates `jackson-core` from 2.8.11 to 2.9.9
- [Release notes](https://github.com/FasterXML/jackson-core/releases)
- [Commits](https://github.com/FasterXML/jackson-core/compare/jackson-core-2.8.11...jackson-core-2.9.9)

Updates `jackson-databind` from 2.8.11 to 2.9.9
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

Updates `jackson-annotations` from 2.8.11 to 2.9.9
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

3 years agoDRILL-6974: Three new tests for 'set' command (#585)
Pavel Semenov [Wed, 22 May 2019 14:18:43 +0000 (17:18 +0300)] 
DRILL-6974: Three new tests for 'set' command (#585)

3 years agoDRILL-7050 - Testing subqueries in the SELECT statement. (#584)
Denys Ordynskiy [Tue, 14 May 2019 15:53:38 +0000 (18:53 +0300)] 
DRILL-7050 - Testing subqueries in the SELECT statement. (#584)

3 years agoBaseline updates for stats tests (#582)
Anisha V Reddy [Wed, 8 May 2019 22:53:51 +0000 (15:53 -0700)] 
Baseline updates for stats tests (#582)

3 years agoDRILL-7038: Added tests to verify that scan is not happening when only partition...
Anton Gozhiy [Thu, 18 Apr 2019 23:41:32 +0000 (02:41 +0300)] 
DRILL-7038: Added tests to verify that scan is not happening when only partition is being selected (#580)

3 years agoFramework changes and tests for RM queue selection (#571)
Abhishek Ravi [Mon, 15 Apr 2019 20:29:37 +0000 (13:29 -0700)] 
Framework changes and tests for RM queue selection (#571)

* Framework changes and tests for RM queue selection

- Added a new class for RM queue selection tests.
- Added new methods in ConnectionPool to create connections based on user, password, tags and groups.
- Refactored restartDrill out of TestDriver, since it is a utility.
- Create connection with schema, group and queryTags properties.
- Changes in JavaTestBase to get all Drillbits' hostnames.
- Fixed faulty tests in TestSSLProperties

* Cosmetic changes. Add comments for the tests.

* Add package to manage DrillCluster from SSH

- Added classes to manage SSH sessions from a user to a DrillCluster.
- Most of the code is re-used from mapr/ycsb-driver implementations.
- The DrillCluster instance should be used when the test framework has to talk to the cluster for copying files or running certain commands.
- Modified the test cases accordingly.
- Added ThrowingConsumer to support lambda for functions with checked exceptions.

* Cosmetic changes, add javadoc for new utility methods

* Existing TestDriver continue using older method

* Small change, missed synchronized

* Add a method for cleaning up cluster before every test

- Remove any existing RM config in DrillCluster.

* Add cleanup after class as well, other cosmetic changes

* Minor change, add description to testng methods

* Run cleanup once for a class

* Cleanup and add test case for unknown tag

3 years agoAdded the Framework version check before the test runs (#574)
Anton Gozhiy [Sat, 13 Apr 2019 22:38:44 +0000 (01:38 +0300)] 
Added the Framework version check before the test runs (#574)

It is useful when analyzing Jenkins test job results

3 years agoAdded a general command line parameter "--jvm" to pass Java options to the test runs...
Anton Gozhiy [Fri, 12 Apr 2019 21:44:55 +0000 (00:44 +0300)] 
Added a general command line parameter "--jvm" to pass Java options to the test runs and --debug to enable remote debug with default options (#573)

The following examples both enable remote debug during the test run:

./bin/run_tests -s Functional -g functional -d --jvm "-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=50000,server=y,suspend=y"

./bin/run_tests -s Functional -g functional -d --debug

3 years agoDisable a stats test case (#578)
Anisha V Reddy [Fri, 12 Apr 2019 20:39:31 +0000 (13:39 -0700)] 
Disable a stats test case (#578)

3 years agoChanges for calculating "optimal memory requirement" for RM (#576)
Abhishek Ravi [Fri, 12 Apr 2019 20:19:47 +0000 (13:19 -0700)] 
Changes for calculating "optimal memory requirement" for RM (#576)

* Changes for calculating "optimal memory requirement" for RM

- Added utilities to get the total memory required per operator
- To get the total memory required by the query
- Get the different operators in the query

* Update unit test to fail in case of exception

3 years agoUpdated the schema name in generation scripts (#577)
Denys Ordynskiy [Thu, 11 Apr 2019 06:23:37 +0000 (09:23 +0300)] 
Updated the schema name in generation scripts (#577)

3 years agoDRILL-7063: Update addremove_files.sh (#575)
Abhishek Girish [Wed, 10 Apr 2019 20:04:45 +0000 (13:04 -0700)] 
DRILL-7063: Update addremove_files.sh (#575)

3 years agoTrivial fix for regex in stats tests (#572)
Anisha V Reddy [Fri, 5 Apr 2019 18:34:41 +0000 (11:34 -0700)] 
Trivial fix for regex in stats tests (#572)

3 years ago Drill RM configuration changes - Read / Modify / Write configuration (#557)
Abhishek Ravi [Fri, 5 Apr 2019 17:46:30 +0000 (10:46 -0700)] 
 Drill RM configuration changes - Read / Modify / Write configuration (#557)

* Drill RM configuration changes - Read / Modify / Write configuration

Initial code for RM related changes.
- This PR introduces a DrillConfigRenderer interface that should be implemented by an class that represents a Drill configuration and requires the ability to serialize its instance to HOCON format.
- Added DrillRMConfig which represents RM configuration in Java. This can be used read RM config into memory, modify RM config and update the configuration.
- Moved the DrillTestFrameworkUnitTests out of resourcemanagement package.
- Added a sample RM config file to test resources.

* Refactor, cleanup and Unit tests

- Refactored code as a part of cleanup.
- Added unit tests for DrillRMConfig.

TODO:
- Add tests to verify serializer to HOCON

* Add unit tests for DrillRMConfig#render

* Changes to get Drillbit hostnames

- Added utility to get the hostnames on which Drillbits are running on the cluster.
- Added toString to CmdConsOut for better debuggability

* Add utility to apply the RM Config to specified drillbit

* Use TEST_ROOT_DIR instead of CWD

- When running tests from testng, CWD points to rootDir/framework.
- Adding a defualt TEST_ROOT_DIR which always points to root directory.

* Cleanup and add unit tests for the changes.

3 years agoTests partitioned files and more query plan verification tests (#570)
Anisha V Reddy [Fri, 5 Apr 2019 17:32:23 +0000 (10:32 -0700)] 
Tests partitioned files and more query plan verification tests (#570)

3 years agoAdded tests to check Drill queries to the Amazon S3 API via Minio (#566)
Denys Ordynskiy [Fri, 5 Apr 2019 16:13:50 +0000 (19:13 +0300)] 
Added tests to check Drill queries to the Amazon S3 API via Minio (#566)

3 years agoMD-5043 Disable s3 storage plugin if Minio is down. Move start_minio script outside...
Denys Ordynskiy [Tue, 2 Apr 2019 17:35:33 +0000 (20:35 +0300)] 
MD-5043 Disable s3 storage plugin if Minio is down. Move start_minio script outside of the gen scripts block. (#568)

3 years agoDRILL-4858: Added tests to verify that repeated_count function can work with repeated...
Anton Gozhiy [Fri, 29 Mar 2019 17:48:19 +0000 (19:48 +0200)] 
DRILL-4858: Added tests to verify that repeated_count function can work with repeated lists and maps (#569)

3 years agoDRILL-7130: Add test for the issue (#567)
Abhishek Ravi [Thu, 28 Mar 2019 00:12:57 +0000 (20:12 -0400)] 
DRILL-7130: Add test for the issue (#567)

- The issue reported in DRILL-7130 can be easily reproduced by having a table with 100 rows and 1 column (of 100 bytes). Add a single row of size < than 100 bytes.
- Query the parquet file.

3 years agoMD-5043 added tstbucket folder to fix the Minio server (#565)
Denys Ordynskiy [Wed, 27 Mar 2019 18:21:03 +0000 (20:21 +0200)] 
MD-5043 added tstbucket folder to fix the Minio server (#565)

3 years agoMD-5043 Added minio server to test S3 API in the drill-test-framework (#554)
Denys Ordynskiy [Wed, 27 Mar 2019 15:51:54 +0000 (17:51 +0200)] 
MD-5043 Added minio server to test S3 API in the drill-test-framework (#554)

3 years agoIntegrate TestNG to enable Java-based test case development & Parse QueryProfile...
Abhishek Ravi [Tue, 26 Mar 2019 18:20:17 +0000 (14:20 -0400)] 
Integrate TestNG to enable Java-based test case development & Parse QueryProfile (#551)

* Integrate TestNG with framework

- Add testng dependency to pom and define a new build profile.
- Added a base class for all Java tests.
- Added a sample test class for RM.

* Add support to get and parse query profile

- Update the REST utils used for Http GET.
- Add utilility to build Http GET request.
- Add utility method to convert the Http Response to a String response.
- Add utility that takes in a queryId and returns a QueryProfile object.

* Handle queryId does not exist scenario

- Check for scenario when Http GET request comes back with an error.
- Throw meaningful error if GET request fails.

* Move createConnectionProperties to Utils, create a class for unit tests

- Test Framework also needs unit tests to ensure quality and robustness
- Move static class createConnectionProperties to Utils.java

* Move debugging logs to debug

* Cosmetic changes

* Reverting the CWD fix to enable merging the code.

* Reverting comment

* Rename class to DrillTestNGDefaults

* Implement review comments - reformat code

* Remove unnecessary blank lines

3 years agoRemove custom logs in CmdConsOut, Add consoleErr for error logs (#561) (#562) (#563)
Abhishek Ravi [Tue, 26 Mar 2019 15:07:57 +0000 (11:07 -0400)] 
Remove custom logs in CmdConsOut, Add consoleErr for error logs (#561) (#562) (#563)

* Remove custom logs in CmdConsOut

- Changes to remove custom logging from CmdConsOut so that the returned result can be parsed, if needed.
- Corresponding changes in the framework

* Add toString to CmdConsOut for logging

* Capture error messages on stdout as well

- For certain commands, the error messages are thrown on stdout. Change here is to capture messages in stdout and stderr when exitcode is not 0.

* Revert local logging change

* Implement review comments - change field names

* Implement code review comments - trivial change - remove tabs

* Skip output / error when empty

3 years agoDRILL-6976: Added tests to verify that an empty function result in a subquery no...
Anton Gozhiy [Tue, 26 Mar 2019 14:38:23 +0000 (16:38 +0200)] 
DRILL-6976: Added tests to verify that an empty function result in a subquery no longer throws an exception. (#564)

3 years agoStatistics: Tests to verify simple query plans (#558)
Anisha V Reddy [Mon, 18 Mar 2019 23:10:26 +0000 (16:10 -0700)] 
Statistics: Tests to verify simple query plans (#558)

3 years agoDRILL-6962: Added tests to verify that coalesce function doesn't throw exception...
Anton Gozhiy [Mon, 18 Mar 2019 15:33:38 +0000 (17:33 +0200)] 
DRILL-6962: Added tests to verify that coalesce function doesn't throw exception if use it with non-existing columns. (#550)

3 years agoDRILL-6962: Added handling of the NULL column type in result set as a workaround...
Anton Gozhiy [Mon, 18 Mar 2019 15:32:59 +0000 (17:32 +0200)] 
DRILL-6962: Added handling of the NULL column type in result set as a workaround of DRILL-7084 (#560)

3 years agoEnable stats tests (#555)
Anisha V Reddy [Tue, 12 Mar 2019 22:54:05 +0000 (15:54 -0700)] 
Enable stats tests (#555)

Stats tests passing as mentioned above.

3 years agoChanges in the pattern of Uncategorized Exception Error (#552)
Kedar Sankar Behera [Tue, 12 Mar 2019 22:29:37 +0000 (15:29 -0700)] 
Changes in the pattern of Uncategorized Exception Error (#552)

3 years agoRename stats.json to stats.json.fail (#553)
Kedar Sankar Behera [Sat, 9 Mar 2019 05:11:19 +0000 (21:11 -0800)] 
Rename stats.json to stats.json.fail (#553)

Removing stat tests until it is stable from drill side

3 years agoCatregorization of Exception Failures (#533)
Kedar Sankar Behera [Fri, 8 Mar 2019 00:36:03 +0000 (16:36 -0800)] 
Catregorization of Exception Failures (#533)

* Initial changes for the exception JIRA

* Co-Authored-By: kedarbcs16 <kedar.behera@gmail.com>
* Changes to address the comments

3 years agoFixes for stats regression failures (#549)
Anisha V Reddy [Wed, 6 Mar 2019 00:54:13 +0000 (16:54 -0800)] 
Fixes for stats regression failures (#549)

3 years agoRevert "Revert "Tests for new table stats feature (#537)" (#543)" (#548)
Kedar Sankar Behera [Mon, 4 Mar 2019 23:47:54 +0000 (15:47 -0800)] 
Revert "Revert "Tests for new table stats feature (#537)" (#543)" (#548)

This reverts commit 25679fae87a3ea38186ddc7f6cdb5a89687ea0e7.

3 years agoMissed enabling a test case for runtime filter (#547)
Abhishek Ravi [Wed, 27 Feb 2019 21:24:32 +0000 (13:24 -0800)] 
Missed enabling a test case for runtime filter (#547)

3 years agoEnable tests for runtime filter since bugs are resolved. (#546)
Abhishek Ravi [Tue, 26 Feb 2019 19:25:53 +0000 (11:25 -0800)] 
Enable tests for runtime filter since bugs are resolved. (#546)

3 years agoSemi Join Testcases (#534)
Kedar Sankar Behera [Thu, 21 Feb 2019 20:10:53 +0000 (12:10 -0800)] 
Semi Join Testcases (#534)

3 years agoCross join Testcases (#535)
Kedar Sankar Behera [Thu, 21 Feb 2019 20:10:16 +0000 (12:10 -0800)] 
Cross join Testcases (#535)

3 years agoHandle map NPE to meaningful IOException (#544)
Abhishek Ravi [Thu, 21 Feb 2019 01:35:18 +0000 (17:35 -0800)] 
Handle map NPE to meaningful IOException (#544)

* MD-5532: Handle map NPE to meaningful IOException

- Handle NullPointerException and throw the right exception with meaningful cause.
- Remove bad formatting in associated methods.

* Reverting re-format changes

3 years agoRevert "Tests for new table stats feature (#537)" (#543)
Abhishek Girish [Sat, 16 Feb 2019 00:49:46 +0000 (16:49 -0800)] 
Revert "Tests for new table stats feature (#537)" (#543)

This reverts commit c2d92a0bbd2965c37b5a65cfb210824e16638165.

3 years agoTests for new table stats feature (#537)
Anisha V Reddy [Thu, 14 Feb 2019 20:28:29 +0000 (01:58 +0530)] 
Tests for new table stats feature (#537)

3 years agoDRILL-6744: Added tests to verify that filter pushdown works with var… (#539)
Anton Gozhiy [Sat, 9 Feb 2019 03:21:12 +0000 (05:21 +0200)] 
DRILL-6744: Added tests to verify that filter pushdown works with var… (#539)

* DRILL-6744: Added tests to verify that filter pushdown works with varchar and decimal data types

Covered:
- Varchar and decimals with all physical types (int32, int64, fixed_len_byte_array, binary).
- Parquet files version 1.8.1 and 1.10.0.
- Queries with and without metadata files.
- Metadata generated by Drill 1.14.0 and 1.16.0-SNAPSHOT.
- Queries on Hive tables.
Dataset:
- Tables were generated from the TPCH data with scale 0.01 with decimal columns and small row group size.
- External Hive tables.

* DRILL-6744: Addressed the review comments

3 years agoDRILL-6964: Baseline updates for implementing CREATE / DROP SCHEMA commands (#541)
Denys Ordynskiy [Thu, 31 Jan 2019 23:25:56 +0000 (01:25 +0200)] 
DRILL-6964: Baseline updates for implementing CREATE / DROP SCHEMA commands (#541)

3 years agoBaseline update due to changes in DRILL-6910 (#538)
Anton Gozhiy [Fri, 25 Jan 2019 18:09:15 +0000 (20:09 +0200)] 
Baseline update due to changes in DRILL-6910 (#538)

3 years agoDRILL-1248: Add support for using aliases in group by (#536)
Denys Ordynskiy [Tue, 22 Jan 2019 22:14:27 +0000 (00:14 +0200)] 
DRILL-1248: Add support for using aliases in group by (#536)

3 years agoDRILL-6797: Added tests to verify that split function no longer returns an error...
Anton Gozhiy [Tue, 22 Jan 2019 18:20:49 +0000 (20:20 +0200)] 
DRILL-6797: Added tests to verify that split function no longer returns an error when result has no rows (#531)

3 years agoDRILL-3610: Functional tests for TIMESTAMPADD and TIMESTAMPDIFF (#530)
Abhishek Ravi [Wed, 16 Jan 2019 17:25:41 +0000 (09:25 -0800)] 
DRILL-3610: Functional tests for TIMESTAMPADD and TIMESTAMPDIFF (#530)

* DRILL-3610: Functional tests for TIMESTAMPADD and TIMESTAMPDIFF functions

- Basic functional tests for TIMESTAMPADD and TIMESTAMPDIFF functions.
- Tests cover different SQL_TSI intervals.

* DRILL 3610: Address review comments, make it self-sufficient

3 years agoDetect DRILL_VERSION from git.properties (#528)
Kunal Khatua [Thu, 10 Jan 2019 19:16:42 +0000 (11:16 -0800)] 
Detect DRILL_VERSION from git.properties (#528)

This would address issues arising out of 4 digit versioning of Drill or any other similar changes (e.g. version...-SNAPSHOT)

3 years agoFix failures in JPPD tests (#526)
Abhishek Ravi [Fri, 28 Dec 2018 00:55:44 +0000 (16:55 -0800)] 
Fix failures in JPPD tests (#526)

* Fix plan verification failures due to Semi Join

- In case of nested join scenario, multiple RTFs arrives at scan on probe side. This resulted in plan showing multiple RuntimeFilters. This has been fixed and hence changes are required.

- Semi Join is enable by default and the plan has to factor in the changes.

- Disable Semi Join where Agg is required before Join.

* Disable failing queries with a reference to the bug

3 years agoDRILL-6865: Added tests to verify the bug fix. (#524)
Anton Gozhiy [Thu, 27 Dec 2018 18:45:17 +0000 (20:45 +0200)] 
DRILL-6865: Added tests to verify the bug fix. (#524)

Cases covered:
1. Filter pruning shouldn't work if at least one filter cannot be pruned ('like' operator or expression on a column).
2. Filter pruning should work in case of single parquet file with single row group.

3 years agoFixed baselines due to the renamed option (#525)
Anton Gozhiy [Mon, 24 Dec 2018 20:15:29 +0000 (22:15 +0200)] 
Fixed baselines due to the renamed option (#525)

exec.return_result_set_for_ddl -> exec.query.return_result_set_for_ddl

3 years agoAdvanced data verification and plan verification tests for RuntimeFilter (#514)
Abhishek Ravi [Wed, 19 Dec 2018 22:45:07 +0000 (04:15 +0530)] 
Advanced data verification and plan verification tests for RuntimeFilter (#514)

* Advanced data verification and plan verification tests for RuntimeFilter

Runtime filter was added to DRILL as a part of DRILL-6385, DRILL-6731 and DRILL-6792.

This PR contains the following changes

- Data verification tests for broadcast and hash joins with join on different data types.
- Plan verification tests to verify Runtime Filter is applied on probe side scan, wherever applicable.
- The queries are run against TPC-H SF 100 data

* Minor fix, remove tsv header for nested broadcast scenario

* Incorporated review comments

- Changed .q and .q.explain extensions to .sql and .sql.explain.
- Explictly enable / disable broadcast join and set the threshold so that change in default values for these parameters will not affect the tests.
- Round decimals to 3 digits to have more stable test case verification.
- Remove "alter session" to conform with standard for setting / resetting session level parameters.

3 years agoDRILL-6834: Added tests to verify Drill option exec.return_result_set_for_ddl (#523)
Anton Gozhiy [Fri, 14 Dec 2018 02:04:45 +0000 (04:04 +0200)] 
DRILL-6834: Added tests to verify Drill option exec.return_result_set_for_ddl (#523)

3 years agoDRILL-6834: Added possibility to run queries that don't return result set. (#521)
Anton Gozhiy [Wed, 12 Dec 2018 18:24:46 +0000 (20:24 +0200)] 
DRILL-6834: Added possibility to run queries that don't return result set. (#521)

An option "exec.return_result_set_for_ddl" was introduced in Drill, when it is false, some queries don't return result set, that cause an SQL exception in Drill Test Framework.
To fix that, Statement execute() method now used instead of executeQuery(). Also added required null checks.

3 years agoScript change in import table to delete existing table before creating a new one...
Kedar Sankar Behera [Tue, 11 Dec 2018 18:55:16 +0000 (10:55 -0800)] 
Script change in import table to delete existing table before creating a new one (#522)

3 years agoDRILL-5796: Added filter pushdown tests for files with multiple row groups. (#519)
Anton Gozhiy [Tue, 11 Dec 2018 17:33:01 +0000 (19:33 +0200)] 
DRILL-5796: Added filter pushdown tests for files with multiple row groups. (#519)

Before changes:
```
Functional/filter/pushdown
                  |_data
                  |_plan
                     |_regex
                     |_regex_no_order
```
After changes:
```
Functional/filter/pushdown
                  |_boolean_null
                  |_cast
                  |_complex_fields
                  |_general
                  |_item_star_operator
                  |_multiple_row_groups
                  |_union
```
Details:
* Added new tests to multiple_row_groups folder. They verify that filter pushdown is working for parquet files with more than one row group.
* Moved existing tests to separate folders depending on a feature tested.
* Combined data and plan verifications within single directories.
* Moved schema definitions from the queries to the json files (like dfs.drillTestDir).
* Changed extension of some tests to match the guidelines (*.sql.explain, *.plan)

3 years agoRemoving blank lines in sql scripts (#520)
Kedar Sankar Behera [Thu, 6 Dec 2018 04:57:42 +0000 (20:57 -0800)] 
Removing blank lines in sql scripts (#520)

3 years agoRemoved newlines in gen scripts as Sqlline 1.6.0 handles them incorrectly. (#518)
Anton Gozhiy [Mon, 3 Dec 2018 18:25:17 +0000 (20:25 +0200)] 
Removed newlines in gen scripts as Sqlline 1.6.0 handles them incorrectly. (#518)

3 years agoSet `store.hive.parquet.optimize_scan_with_native_reader` to true to prevent (#500)
Robert Hou [Wed, 21 Nov 2018 00:38:17 +0000 (16:38 -0800)] 
Set `store.hive.parquet.optimize_scan_with_native_reader` to true to prevent (#500)

some TPCDS tests from failing when using hive reader until HIVE-19069 is fixed.

3 years agoBaseline updates in Hive Mapr DB json tests. (#516)
Anton Gozhiy [Sat, 17 Nov 2018 00:32:16 +0000 (02:32 +0200)] 
Baseline updates in Hive Mapr DB json tests. (#516)

3 years agoFixed baselines due to changes in DRILL-6744 (#515)
Anton Gozhiy [Fri, 16 Nov 2018 18:19:03 +0000 (20:19 +0200)] 
Fixed baselines due to changes in DRILL-6744 (#515)

3 years agoMD-3355: baseline updates for MapR-DB tests (#513)
Denys Ordynskiy [Tue, 6 Nov 2018 19:32:15 +0000 (21:32 +0200)] 
MD-3355: baseline updates for MapR-DB tests (#513)

3 years agoFixed failures caused by impersonation. (#512)
Anton Gozhiy [Fri, 2 Nov 2018 17:38:33 +0000 (19:38 +0200)] 
Fixed failures caused by impersonation. (#512)

Also, a baseline fix is included.

3 years agoMD-5018: Copy csv to dfs for every run (#511)
Abhishek Ravi [Fri, 2 Nov 2018 16:58:23 +0000 (09:58 -0700)] 
MD-5018: Copy csv to dfs for every run (#511)

* MD-5018: Replace DRILL_5941_lines_count_skip_header_footer_big_data_set.csv every time

- The existing script does not copy `DRILL_5941_lines_count_skip_header_footer_big_data_set.csv` to dfs location if it already exists. It is possible that previous copy did not go through completely and we may have a corrupt data. This could result in tests failing.
- The fix is to copy csv every time.

* MD-5018: Addressing review comments, untar every run

3 years agoUpdating Jackson lib to 2.8.11 (#510)
Abhishek Girish [Wed, 24 Oct 2018 06:47:30 +0000 (23:47 -0700)] 
Updating Jackson lib to 2.8.11 (#510)

3 years agoDo not exit on failure (#508)
Abhishek Ravi [Tue, 16 Oct 2018 23:40:36 +0000 (16:40 -0700)] 
Do not exit on failure (#508)

Drop tables may fail if tables do not exist, do not exit script on error.

3 years agoUpdating bin/run_tests to ignore MapR minor version mismatch (#507)
Abhishek Girish [Mon, 15 Oct 2018 16:24:55 +0000 (12:24 -0400)] 
Updating bin/run_tests to ignore MapR minor version mismatch (#507)

3 years agoAdding support for MapR-FS client as a maven dependency (#506)
Abhishek Girish [Thu, 11 Oct 2018 00:28:27 +0000 (17:28 -0700)] 
Adding support for MapR-FS client as a maven dependency (#506)

3 years agoDRILL-6624: Adding tests for back-ticks in schema + workspace (#505)
Denys Ordynskiy [Wed, 19 Sep 2018 03:27:21 +0000 (05:27 +0200)] 
DRILL-6624: Adding tests for back-ticks in schema + workspace (#505)

3 years agoDRILL-6612: Inner join temporary and permanent tables (#503)
Denys Ordynskiy [Fri, 14 Sep 2018 16:11:37 +0000 (19:11 +0300)] 
DRILL-6612: Inner join temporary and permanent tables (#503)

3 years agoDRILL-5188: Added TPC-DS queries that were previously failing due to IllegalArgumentE...
Anton Gozhiy [Fri, 14 Sep 2018 16:10:20 +0000 (19:10 +0300)] 
DRILL-5188: Added TPC-DS queries that were previously failing due to IllegalArgumentException (#504)

3 years agoDRILL-6710: baseline updates for decimal type scale enhancements (#501)
Denys Ordynskiy [Wed, 12 Sep 2018 00:02:06 +0000 (03:02 +0300)] 
DRILL-6710: baseline updates for decimal type scale enhancements (#501)