drill.git
11 hours agoDRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints (#2549) master
Niels Basjes [Thu, 19 May 2022 00:57:08 +0000 (02:57 +0200)] 
DRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints (#2549)

* DRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints

* DRILL-8225: Replace Caffeine caching with Java 8 compliant solution

* DRILL-8225: Fix dependency issues

* DRILL-8225: Use new API for JDK8 caching

41 hours agoDRILL-8223: Refactor auth modes dropping DRILL_PROCESS and allowing credential provid...
James Turton [Tue, 17 May 2022 18:58:05 +0000 (20:58 +0200)] 
DRILL-8223: Refactor auth modes dropping DRILL_PROCESS and allowing credential providers everywhere (#2547)

* Remove CredentialedStoragePluginConfig and drill_process auth mode.

* Add a unit test of shared_user with no creds based on H2.

* Fix compile errors.

2 days agoDRILL-8220: Add User Translation Support for OAuth Enabled Plugins (#2544)
Charles S. Givre [Mon, 16 May 2022 14:09:10 +0000 (10:09 -0400)] 
DRILL-8220: Add User Translation Support for OAuth Enabled Plugins (#2544)

3 days agoDRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (set hadoop.version...
James Turton [Mon, 16 May 2022 10:54:18 +0000 (12:54 +0200)] 
DRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (set hadoop.version) (#2550)

5 days agoDRILL-8224: Fix TestHttpPlugin#testSlowResponse
Vitalii Diravka [Thu, 12 May 2022 17:34:15 +0000 (20:34 +0300)] 
DRILL-8224: Fix TestHttpPlugin#testSlowResponse

5 days agoDRILL-8214: Replace EnumerableTableScan usage with LogicalTableScan (#2538)
Volodymyr Vysotskyi [Fri, 13 May 2022 18:06:28 +0000 (21:06 +0300)] 
DRILL-8214: Replace EnumerableTableScan usage with LogicalTableScan (#2538)

6 days agoDRILL-8211: Replace deprecated RelNode.getChildExps with Project.getProjects (#2535)
Volodymyr Vysotskyi [Thu, 12 May 2022 17:27:55 +0000 (20:27 +0300)] 
DRILL-8211: Replace deprecated RelNode.getChildExps with Project.getProjects (#2535)

7 days agoDRILL-8216: Use EVF-based JSON reader for Values operator
Volodymyr Vysotskyi [Sun, 8 May 2022 10:54:45 +0000 (13:54 +0300)] 
DRILL-8216: Use EVF-based JSON reader for Values operator

7 days agoDRILL-8218: Add unit tests for StorageResource REST endpoints (#2541)
James Turton [Thu, 12 May 2022 04:48:20 +0000 (06:48 +0200)] 
DRILL-8218: Add unit tests for StorageResource REST endpoints (#2541)

7 days agoDRILL-8222: Fix wrong func impl of concat_delim, when null-value exist in middle...
xurenhe [Wed, 11 May 2022 13:16:11 +0000 (21:16 +0800)] 
DRILL-8222: Fix wrong func impl of concat_delim, when null-value exist in middle args (#2546)

8 days agoDRILL-8219: Handle null catalog names returned by DB2 in storage-jdbc. (#2542)
James Turton [Tue, 10 May 2022 16:22:57 +0000 (18:22 +0200)] 
DRILL-8219: Handle null catalog names returned by DB2 in storage-jdbc. (#2542)

9 days agoDRILL-8210: Add substring convertlet
Volodymyr Vysotskyi [Sat, 7 May 2022 15:32:20 +0000 (18:32 +0300)] 
DRILL-8210: Add substring convertlet

9 days agoDRILL-8213: Replace deprecated RelNode.getRows with RelNode.estimateRowCount
Volodymyr Vysotskyi [Sat, 7 May 2022 16:29:41 +0000 (19:29 +0300)] 
DRILL-8213: Replace deprecated RelNode.getRows with RelNode.estimateRowCount

9 days agoDRILL-8212: Join queries fail with StackOverflowError (#2536)
Volodymyr Vysotskyi [Mon, 9 May 2022 15:52:24 +0000 (18:52 +0300)] 
DRILL-8212: Join queries fail with StackOverflowError (#2536)

10 days agoDRILL-8215: Remove SecurityContext from PluginConfigWrapper
Charles S. Givre [Mon, 9 May 2022 09:05:05 +0000 (05:05 -0400)] 
DRILL-8215: Remove SecurityContext from PluginConfigWrapper

10 days agoDRILL-8208: Create builder for SqlSelect (#2532)
Volodymyr Vysotskyi [Mon, 9 May 2022 08:46:06 +0000 (11:46 +0300)] 
DRILL-8208: Create builder for SqlSelect (#2532)

12 days agoDRILL-8207: Fix Username Typo and password @JsonIgnore in JDBC SerDe (#2531)
James Turton [Fri, 6 May 2022 15:42:59 +0000 (17:42 +0200)] 
DRILL-8207: Fix Username Typo and password @JsonIgnore in JDBC SerDe (#2531)

13 days agoDRILL-8207: Fix typo in JDBC config (#2530)
Charles S. Givre [Thu, 5 May 2022 21:33:40 +0000 (17:33 -0400)] 
DRILL-8207: Fix typo in JDBC config (#2530)

2 weeks ago[MINOR UPDATE] Update AWS Java SDK to 1.12.211
Charles S. Givre [Thu, 5 May 2022 07:13:28 +0000 (03:13 -0400)] 
[MINOR UPDATE] Update AWS Java SDK to 1.12.211

2 weeks agoDRILL-8204: Allow Provided Schema for HTTP Plugin in JSON Mode (#2526)
Charles S. Givre [Tue, 3 May 2022 20:32:25 +0000 (16:32 -0400)] 
DRILL-8204: Allow Provided Schema for HTTP Plugin in JSON Mode (#2526)

* Initial commit

* Map working

* WIP

* Added builder

* Lists in maps working

* Add documentation

* Cleaned up UT

* Final Revision

* Fix checkstyle

* Minor tweak

* removed extra test file

* Removed unused import

* Added inline schema support

* Addressed review comments

* Removed unused import

* Removed json string

* Final Revisions

* Fixed unit test

2 weeks ago[MINOR UPDATE] Fix default auth mode in AuthMode.parseOrDefault and test the case...
James Turton [Tue, 3 May 2022 14:20:33 +0000 (16:20 +0200)] 
[MINOR UPDATE] Fix default auth mode in AuthMode.parseOrDefault and test the case of JDBC. (#2528)

2 weeks agoDRILL-8035: Update Janino to 3.1.7 version
Volodymyr Vysotskyi [Sat, 5 Mar 2022 08:57:57 +0000 (10:57 +0200)] 
DRILL-8035: Update Janino to 3.1.7 version

2 weeks agoDRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (#2525)
James Turton [Sat, 30 Apr 2022 09:21:40 +0000 (11:21 +0200)] 
DRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (#2525)

* Remove pointless Buffer casts.

Compiling Drill with JDK > 8 will still result in ByteBuffer <-> Buffer cast
exceptions at runtime when running on JDK 8 even though maven.target.version
is set to 8. Setting maven.compiler.release to 8 solves the Buffer casts
but raises a compilation error of package sun.security.jgss does not exist
for JDK 8. There were a few handwritten casts to avoid the Buffer casting
issue but many instances are not covered so the few reverted in this commit
achieve nothing.

* Update Hadoop to 3.2.3.

2 weeks agoDRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin (#2524)
Charles S. Givre [Thu, 28 Apr 2022 14:33:13 +0000 (10:33 -0400)] 
DRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin (#2524)

* DRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin

* Fixed typo

3 weeks agoDRILL-8195: Add Timestamp Zone offset ISO-8601 format for JSON EVF
Vitalii Diravka [Tue, 26 Apr 2022 14:09:02 +0000 (17:09 +0300)] 
DRILL-8195: Add Timestamp Zone offset ISO-8601 format for JSON EVF

3 weeks agoDRILL-8037: Add V2 JSON Format Plugin based on EVF
Vitalii Diravka [Fri, 5 Nov 2021 02:21:40 +0000 (04:21 +0200)] 
DRILL-8037: Add V2 JSON Format Plugin based on EVF

* Enable store.json.enable_v2_reader by default
* Fix TestJsonReader doulbe quotes test cases. Update jackson 2.12.1 -> 2.13.0
* Disable V2 for experimental UNION datatype
* Fix regressions
* Fix json Schema Provision (it wasn't provided for JsonLoaderBuilder). The previous schema provision was a fake, the reader schema was infered from the json content. It fixes the scan and reader schema validation. And it starts to apply the provided schema to ANALYZE COMMANDS, fixed TestMetastoreWithEasyFormatPlugin#testAnalyzeOnJsonTable

3 weeks agoDRILL-6953: Merge row set-based JSON reader
Paul Rogers [Tue, 14 Apr 2020 02:45:16 +0000 (19:45 -0700)] 
DRILL-6953: Merge row set-based JSON reader

* Snapshot: reworked JSON field parser creation
* Updated JSON loader
* Redo value listener with tokens
* Extended long type works
* More simple extended types
* Added $date
* Binary type
* All extended type except arrays
* Extended arrays partly working
* More arrays work
* Refactor element parser interfaces
* Rename RowSetTests --> RowSetTest
* More factory cleanup
* Revised unknown field creation
* In middle of factory/parser restructuring
* Scalars, object, some variants work again
* JSON loader tests pass
* File cleanup
* Old extended types test passes
* Renamed JSON packages
* Tested extended provided types

3 weeks agoDRILL-8155: Introduce New Plugin Authentication Modes (#2516)
Charles S. Givre [Wed, 27 Apr 2022 00:15:37 +0000 (20:15 -0400)] 
DRILL-8155: Introduce New Plugin Authentication Modes (#2516)

* Do not set the read-only hint on JDBC connections.

* Outline of different auth modes in storage-jdbc.

In this commit, a new `authMode` storage config supporting three new auth modes
is defined: shared user (default), user translation (user is translated to some
other user from the external storage) and impersonation (the external storage
and JDBC driver provide support for impersonating the Drill query user).

The JdbcStoragePlugin is enhanced to be able to work with a lookup table of
connection pools, where a pool is dedicated to each query user except when
in shared user auth mode.

Planning and execution time APIs are also enhanced to transmit a user
credentials object for the query user, instead of just its username.
This allows for the expansion of the UserCredentials protobuf type to
include some optional extra credentials, e.g. in an array of byte arrays.
These credentials may be relevant in the user tranlsation mode when a
credential provider must be accessed in order to obtain the creds to be used
for the external system.

* Rebased to current master and build fixes

* Build works, cred stuff added

* Credentials being saved and pushed down to storage plugin

* UI now closing properly

* User Translation working for HTTP plugin

* HTTP unit tests passing

* WIP

* Fixed import

* User Credentials now being stored in credential provider

* Working

* Fixed TPCH Unit Tests

* Fix CredProvider SerDe Test

* Added unit tests for JDBC

* Code cleanup

* Fix LGTM alerts

* Correct username now populating Group Scan

* Username to Subscan

* Remove PerUserUsernamePasswordCredentials class.

* Remove getUserCredentials from CredentialsProvider.

* Planning errors fixed

* Removed unused imports

* Fixed minor issues

* Unit test fixes

* WIP.

* Fix CodeQL Alert

* Ignore LGTM False Positive

* Fix tainted string LGTM alert

* Revert LGTM Comment

* Addressed review comments

* Use fixed size Guava caches in JDBC convetion and dialect factories.

These replace Maps with no size limit that might have grown without
bound. LRU eviction begins when the cache size limit is reached.

* Add a TTL to the JDBC dialect and convention caches.

Co-authored-by: James Turton <james@somecomputer.xyz>
3 weeks agoDRILL-8194: Fix the function of REPLACE throws IndexOutOfBoundsException If text...
xurenhe [Sun, 24 Apr 2022 02:44:10 +0000 (10:44 +0800)] 
DRILL-8194: Fix the function of REPLACE throws IndexOutOfBoundsException If text's length is more than previously applied (#2522)

4 weeks agoDRILL-8013: Drill attempts to push "$SUM0" to JDBC storage plugin for AVG (#2521)
Volodymyr Vysotskyi [Wed, 20 Apr 2022 16:27:14 +0000 (19:27 +0300)] 
DRILL-8013: Drill attempts to push "$SUM0" to JDBC storage plugin for AVG (#2521)

5 weeks agoDRILL-8193: Incorrect Annotation used for HttpJsonOptions (#2519)
Charles S. Givre [Wed, 13 Apr 2022 04:01:37 +0000 (00:01 -0400)] 
DRILL-8193: Incorrect Annotation used for HttpJsonOptions (#2519)

5 weeks agoDRILL-8192: Cassandra queries fail when enabled Mongo plugin (#2518)
Volodymyr Vysotskyi [Mon, 11 Apr 2022 00:29:03 +0000 (03:29 +0300)] 
DRILL-8192: Cassandra queries fail when enabled Mongo plugin (#2518)

5 weeks agoDRILL-8191: HTTP Request Function Not Detecting JSON Config (#2517)
Charles S. Givre [Thu, 7 Apr 2022 18:10:37 +0000 (14:10 -0400)] 
DRILL-8191: HTTP Request Function Not Detecting JSON Config (#2517)

6 weeks agoDRILL-8174: Convert Avro format to EVF2 (#2511)
luoc [Mon, 4 Apr 2022 16:21:25 +0000 (00:21 +0800)] 
DRILL-8174: Convert Avro format to EVF2 (#2511)

* DRILL-8174: Convert Avro format to EVF2

* Addressed review comments

6 weeks agoDRILL-8186: Convert XML format to EVF2 (#2512)
luoc [Sun, 3 Apr 2022 21:46:02 +0000 (05:46 +0800)] 
DRILL-8186: Convert XML format to EVF2 (#2512)

6 weeks agoDRILL-8187: Dialect factory returns ANSI SQL dialect for BigQuery (#2513)
Volodymyr Vysotskyi [Sun, 3 Apr 2022 21:45:50 +0000 (00:45 +0300)] 
DRILL-8187: Dialect factory returns ANSI SQL dialect for BigQuery (#2513)

6 weeks agoFix projection of map arrays and nested maps (#2510)
Paul Rogers [Sun, 3 Apr 2022 08:42:27 +0000 (01:42 -0700)] 
Fix projection of map arrays and nested maps (#2510)

Fixes the rather complex area in EVF2 that handles
projection of nested maps and map arrays to properly pass
along the offset vector for map arrays, and set the value
count for nested maps.

6 weeks agoDRILL-8181: Accept nullable args in HTTP plugin UDFs, fix HikariCP default parm names...
James Turton [Thu, 31 Mar 2022 13:49:49 +0000 (15:49 +0200)] 
DRILL-8181: Accept nullable args in HTTP plugin UDFs, fix HikariCP default parm names (#2509)

7 weeks agoDRILL-8176: minor issue in previous jackson bom (#2508)
PJ Fanning [Thu, 31 Mar 2022 12:21:24 +0000 (14:21 +0200)] 
DRILL-8176: minor issue in previous jackson bom (#2508)

* minor issue in previous jackson bom

* Update pom.xml

7 weeks agoDRILL-8177: Bump default TLS version to 1.3 (#2505)
Rymar Maksym [Mon, 28 Mar 2022 18:53:38 +0000 (21:53 +0300)] 
DRILL-8177: Bump default TLS version to 1.3 (#2505)

* Bump default TLS version to 1.3. Fix web server log

* Change TLS version in tests. Change value of TLS constants

7 weeks agoDRILL-8175: Update Drill release script after 1.20 (#2503)
James Turton [Sun, 27 Mar 2022 11:20:27 +0000 (13:20 +0200)] 
DRILL-8175: Update Drill release script after 1.20 (#2503)

* Set DRILL_PID_DIR in Dockerfile to writable location for distributed mode.

Some users of the images built from this Dockerfile customise
them so that they launch Drill in distributed mode instead of
embedded mode.  This change saves them from having to set
DRILL_PID_DIR themselves in order to succeed.

* Update release script and instructions after the release of 1.20.

- Add support for specifying a build profile such as "hadoop-2".
- Update instuctions for the Drill web site.
- Update instructions for uploading RCs (no more home.apache)
- Some fixes.

7 weeks agoDRILL-8178: Bump AWS Libraries to Latest Version (#2506)
Charles S. Givre [Sun, 27 Mar 2022 08:16:51 +0000 (04:16 -0400)] 
DRILL-8178: Bump AWS Libraries to Latest Version (#2506)

7 weeks agoDRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613 (#2493)
luoc [Fri, 25 Mar 2022 18:28:20 +0000 (02:28 +0800)] 
DRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613 (#2493)

* DRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613

* Update the ProfileCopyright tag name

* Update the mov format name

* Add the QuickTime.Rotation tag

* Bump metadata-extractor to 2.17.0

7 weeks agoDRILL-8176: Upgrade Jackson Due to CVE-2020-36518 (#2504)
PJ Fanning [Fri, 25 Mar 2022 18:26:40 +0000 (19:26 +0100)] 
DRILL-8176: Upgrade Jackson Due to CVE-2020-36518 (#2504)

7 weeks ago[MINOR UPDATE] Fix the word spelling error (#2502)
xzh [Fri, 25 Mar 2022 10:27:38 +0000 (18:27 +0800)] 
[MINOR UPDATE] Fix the word spelling error (#2502)

7 weeks ago[MINOR UPDATE] Add license to CodeQL YAML (#2501)
luoc [Fri, 25 Mar 2022 07:07:17 +0000 (15:07 +0800)] 
[MINOR UPDATE] Add license to CodeQL YAML (#2501)

7 weeks agoDRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2 (#2485)
Paul Rogers [Thu, 24 Mar 2022 14:52:19 +0000 (07:52 -0700)] 
DRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2 (#2485)

* DRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2

Also includes:

* DRILL-8159: Convert the HTTPD reader to use EVF V2

* Build fix

* Changes from review comments

* Fix test issue

7 weeks agoUpdate codeql-analysis.yml
Charles S. Givre [Thu, 24 Mar 2022 12:30:10 +0000 (07:30 -0500)] 
Update codeql-analysis.yml

Removed cpp from code analysis

8 weeks agoCreate codeql-analysis.yml
Charles S. Givre [Wed, 23 Mar 2022 22:01:57 +0000 (17:01 -0500)] 
Create codeql-analysis.yml

8 weeks agoDRILL-8171: Convert SequenceFile to EVF2 (#2498)
luoc [Tue, 22 Mar 2022 15:06:10 +0000 (23:06 +0800)] 
DRILL-8171: Convert SequenceFile to EVF2 (#2498)

8 weeks agoDRILL-8165: Upgrade liquibase because of CVE-2022-0839 (#2497)
luoc [Mon, 21 Mar 2022 14:09:59 +0000 (22:09 +0800)] 
DRILL-8165: Upgrade liquibase because of CVE-2022-0839 (#2497)

8 weeks agoDRILL-8172: Use the specified memory usage for Travis CI (#2500)
luoc [Mon, 21 Mar 2022 08:30:57 +0000 (16:30 +0800)] 
DRILL-8172: Use the specified memory usage for Travis CI (#2500)

2 months agoDRILL-8169: Add UDFs to HTTP Plugin to Facilitate Joins (#2496)
Charles S. Givre [Fri, 18 Mar 2022 17:12:28 +0000 (13:12 -0400)] 
DRILL-8169: Add UDFs to HTTP Plugin to Facilitate Joins (#2496)

* Initial commit

* Unit tests wroking

* Added configs from constants

* Working with Storage plugins

* Removed cache files

* Fixed cache directory

* Updated docs

* Removed unneeded pom.xml modifications

* Addressed Review comments

* Converted holder to VarCharHolder[]

2 months agoIncrement the Drill version number to 2.0.0-SNAPSHOT. (#2490)
James Turton [Fri, 18 Mar 2022 16:36:11 +0000 (18:36 +0200)] 
Increment the Drill version number to 2.0.0-SNAPSHOT. (#2490)

2 months agoDRILL-8168: Do not duplicate attempts to impersonate a user in the REST API (#2495)
James Turton [Wed, 16 Mar 2022 10:12:05 +0000 (12:12 +0200)] 
DRILL-8168: Do not duplicate attempts to impersonate a user in the REST API (#2495)

2 months agoDRILL-8167: Add JSON Config Options to Format Config (#2494)
Charles S. Givre [Tue, 15 Mar 2022 12:21:37 +0000 (08:21 -0400)] 
DRILL-8167: Add JSON Config Options to Format Config (#2494)

* Initial work'

* Initial commit

* Added unit tests

* Code cleanup and final param

* Fixed Unit Test

* Addressed Review Comments

2 months agoDRILL-8156: Declare and chown a /data VOLUME in the Drill Dockerfile (#2491)
James Turton [Sun, 13 Mar 2022 15:26:51 +0000 (17:26 +0200)] 
DRILL-8156: Declare and chown a /data VOLUME in the Drill Dockerfile (#2491)

* Add a mountpoint and VOLUME for local storage to Dockerfile.

* Address review comments concerning layer ordering.

* Fix image size blowup by moving chmod to intermediate container.

* Combine RUN commands in Dockerfile.

2 months agoDRILL-8162: Add OpenAPI Specification documentation for Drill's REST API (#2489)
estherbuchwalter [Fri, 11 Mar 2022 21:23:33 +0000 (16:23 -0500)] 
DRILL-8162: Add OpenAPI Specification documentation for Drill's REST API (#2489)

* Excluded swagger dependencies to fix enforcer error

* Added swagger core dependencies and swagger version

* Tested adding swagger annotations

* Added external docs in the openAPI Specification/swagger annotations

* Reverted code reformat. Deencoded URLs. Reformatted log messages.

* More format changes.

2 months agoDRILL-8161: Add Global Credentials to HTTP Storage Plugin (#2486)
Charles S. Givre [Wed, 9 Mar 2022 22:43:08 +0000 (17:43 -0500)] 
DRILL-8161: Add Global Credentials to HTTP Storage Plugin (#2486)

* DRILL-8161: Add Global Credentials to HTTP Storage Plugin

* Removed unused import

* Formatting from code review

* Updated README

2 months ago[MINOR UPDATE] Fix license for Stalebot Config (#2488)
Volodymyr Vysotskyi [Tue, 8 Mar 2022 15:06:54 +0000 (17:06 +0200)] 
[MINOR UPDATE] Fix license for Stalebot Config (#2488)

2 months ago[MINOR UPDATE] Add Stalebot Config (#2487)
Charles S. Givre [Tue, 8 Mar 2022 05:26:53 +0000 (00:26 -0500)] 
[MINOR UPDATE] Add Stalebot Config (#2487)

2 months agoDRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case ...
Vitalii Diravka [Sun, 6 Mar 2022 16:02:12 +0000 (18:02 +0200)] 
DRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case (#2471)

* Set Brotli codec jar and test to occur only on Linux amd64.

* DRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case

- changing timeout for TestDrillbitResilience tests
- timing tuning for memoryLeaksWhenCancelled
- update TestContainers version
- -DforkCount=1 for Travis Maven build
- directMemoryMb: 2500 -> 4500 leads to less occasinal test failures

Co-authored-by: James Turton <james@somecomputer.xyz>
2 months agoDRILL-8154: Upgrade to poi 5.2.1 (#2480)
PJ Fanning [Sat, 5 Mar 2022 12:49:43 +0000 (13:49 +0100)] 
DRILL-8154: Upgrade to poi 5.2.1 (#2480)

2 months agoDRILL-8140: Add JSON Post Body to HTTP Rest Storage Plugin (#2474)
Charles S. Givre [Thu, 3 Mar 2022 18:24:18 +0000 (13:24 -0500)] 
DRILL-8140: Add JSON Post Body to HTTP Rest Storage Plugin (#2474)

* Initial commit

* Addressed Review Comments

* Updated Docs

* Code cleanup per code review

2 months agoDRILL-8151: Add support for more ElasticSearch and Cassandra data types (#2477)
Volodymyr Vysotskyi [Thu, 3 Mar 2022 05:00:23 +0000 (07:00 +0200)] 
DRILL-8151: Add support for more ElasticSearch and Cassandra data types (#2477)

2 months agoDRILL-8153: Convert OAuth REST APIs to JSON #2479
Charles S. Givre [Wed, 2 Mar 2022 23:27:40 +0000 (18:27 -0500)] 
DRILL-8153: Convert OAuth REST APIs to JSON #2479

2 months ago[DRILL-8150] log4j 2.17.2 in format-excel (#2476)
PJ Fanning [Tue, 1 Mar 2022 14:21:07 +0000 (15:21 +0100)] 
[DRILL-8150] log4j 2.17.2 in format-excel (#2476)

2 months agoDRILL-8085: EVF V2 support in the "Easy" format plugin (#2419)
Paul Rogers [Tue, 1 Mar 2022 08:35:57 +0000 (00:35 -0800)] 
DRILL-8085: EVF V2 support in the "Easy" format plugin (#2419)

2 months agoDRILL-8148: Add REST Endpoints to Update OAuth Tokens (#2473)
Charles S. Givre [Mon, 28 Feb 2022 17:45:34 +0000 (12:45 -0500)] 
DRILL-8148: Add REST Endpoints to Update OAuth Tokens (#2473)

* Initial commit

* Added unit tests

* Fixed unused imports

* Addressed Review comments

* Fixed Link Spelling

2 months agoAdd jackson-bom (#2454)
Rymar Maksym [Sun, 27 Feb 2022 00:36:22 +0000 (02:36 +0200)] 
Add jackson-bom (#2454)

2 months agoSAS Reader fixes (#2472)
pseudomo [Sun, 27 Feb 2022 00:35:55 +0000 (01:35 +0100)] 
SAS Reader fixes (#2472)

Co-authored-by: pseudomo <pseudomo@yandex.ru>
2 months ago[maven-release-plugin] prepare for next development iteration
James Turton [Tue, 22 Feb 2022 17:56:15 +0000 (19:56 +0200)] 
[maven-release-plugin] prepare for next development iteration

2 months ago[maven-release-plugin] prepare release drill-1.20.0 drill-1.20.0
James Turton [Tue, 22 Feb 2022 17:56:14 +0000 (19:56 +0200)] 
[maven-release-plugin] prepare release drill-1.20.0

2 months agoDRILL-8144: Cannot launch Drill 1.20 RC 4 on Windows (#2470)
James Turton [Tue, 22 Feb 2022 17:33:51 +0000 (19:33 +0200)] 
DRILL-8144: Cannot launch Drill 1.20 RC 4 on Windows (#2470)

2 months agoDRILL-8143: Error querying json with $date field (#2469)
Anton Gozhiy [Mon, 21 Feb 2022 14:07:38 +0000 (16:07 +0200)] 
DRILL-8143: Error querying json with $date field (#2469)

2 months ago DRILL-8142: SAS Reader Returns NPE #2468
Charles S. Givre [Sun, 20 Feb 2022 05:35:25 +0000 (00:35 -0500)] 
 DRILL-8142: SAS Reader Returns NPE #2468

2 months agoDRILL-8139: Parquet CodecFactory thread safety bug (#2463)
James Turton [Thu, 17 Feb 2022 17:02:20 +0000 (19:02 +0200)] 
DRILL-8139: Parquet CodecFactory thread safety bug  (#2463)

2 months agoDRILL-8134: Cannot query Parquet INT96 columns as timestamps (#2460)
James Turton [Thu, 17 Feb 2022 15:32:27 +0000 (17:32 +0200)] 
DRILL-8134: Cannot query Parquet INT96 columns as timestamps (#2460)

* Scale writer index stride to cater for 96-to-64-bit downcasting.

* Remove advanceWriterIndex methods entirely.

* Bump postgresql in storage-jdbc for dependabot alert.

* Fix config opt setting bug in TestParquetWriter.

3 months agoDRILL-8122: Change kafka metadata obtaining due to KAFKA-5697 (#2456)
Rymar Maksym [Wed, 16 Feb 2022 06:33:29 +0000 (08:33 +0200)] 
DRILL-8122: Change kafka metadata obtaining due to KAFKA-5697 (#2456)

3 months agoDRILL-8137: Prevent reading union inputs after cancellation request (#2462)
Volodymyr Vysotskyi [Wed, 16 Feb 2022 06:32:16 +0000 (08:32 +0200)] 
DRILL-8137: Prevent reading union inputs after cancellation request (#2462)

3 months agoDRILL-8131: Infinite planning when JDBC or Phoenix plugin is enabled (#2459)
Volodymyr Vysotskyi [Sun, 13 Feb 2022 06:11:49 +0000 (08:11 +0200)] 
DRILL-8131: Infinite planning when JDBC or Phoenix plugin is enabled (#2459)

3 months agoDRILL-8130: Upgrade Hadoop 2 to 2.10.1 because of CVE-2020-9492 (#2458)
James Turton [Fri, 11 Feb 2022 08:12:01 +0000 (10:12 +0200)] 
DRILL-8130: Upgrade Hadoop 2 to 2.10.1 because of CVE-2020-9492 (#2458)

* Upgrade Hadoop 2 to 2.10.1 because of CVE-2020-9492.

* Update JDBC jar size limit for Hadoop 2.10.1.

3 months agoDRILL-8129: Storage-phoenix cannot resolve OSGi bundle apache-ds.jdbm1 (#2457)
James Turton [Thu, 10 Feb 2022 04:21:13 +0000 (06:21 +0200)] 
DRILL-8129: Storage-phoenix cannot resolve OSGi bundle apache-ds.jdbm1 (#2457)

* Add maven-bundle-plugin to storage-phoenix pom for -Phadoop-2.

Also correct the jdbc-all jar size limits, esp. hadoop-2 profile.

* Move storage-phoenix maven-bundle-plugin into hadoop-2 profile.

3 months ago DRILL-8126: Ignore OAuth Parameter in Storage Plugin
Charles S. Givre [Tue, 8 Feb 2022 07:41:35 +0000 (02:41 -0500)] 
 DRILL-8126: Ignore OAuth Parameter in Storage Plugin

3 months agoSet failOnNoFiles = false so that checksum-maven-plugin works on tools-parent.
James Turton [Fri, 4 Feb 2022 14:27:41 +0000 (16:27 +0200)] 
Set failOnNoFiles = false so that checksum-maven-plugin works on tools-parent.

3 months agoRevert .travis.yml to 80fa2662.
James Turton [Thu, 3 Feb 2022 12:02:25 +0000 (14:02 +0200)] 
Revert .travis.yml to 80fa2662.

3 months agoFix line continuations in .travis.yml and update NOTICE.
James Turton [Thu, 3 Feb 2022 10:35:30 +0000 (12:35 +0200)] 
Fix line continuations in .travis.yml and update NOTICE.

3 months agoDRILL-8124: Fix implicit file issue with EVF 2 (#2451)
Paul Rogers [Thu, 3 Feb 2022 12:52:27 +0000 (04:52 -0800)] 
DRILL-8124: Fix implicit file issue with EVF 2 (#2451)

3 months agoDRILL-8123: Revise scan limit pushdown (#2450)
Paul Rogers [Thu, 3 Feb 2022 12:51:10 +0000 (04:51 -0800)] 
DRILL-8123: Revise scan limit pushdown (#2450)

3 months agoAdd James Turton's PGP key.
James Turton [Thu, 3 Feb 2022 08:15:25 +0000 (10:15 +0200)] 
Add James Turton's PGP key.

3 months agoDRILL-7994: Dependency version updates for severe vulnerabilities
James Turton [Wed, 2 Feb 2022 15:56:55 +0000 (17:56 +0200)] 
DRILL-7994: Dependency version updates for severe vulnerabilities

Postgresql: 42.3.1 -> 42.3.2.

3 months agoDRILL-7994: Dependency version updates for severe vulnerabilities (#2447)
James Turton [Wed, 2 Feb 2022 14:05:18 +0000 (16:05 +0200)] 
DRILL-7994: Dependency version updates for severe vulnerabilities (#2447)

* Updates deps for dependabot.

* Remove jetty version spec from storage-phoenix pom.

* Revert parallel GC setting.

3 months agoDRILL-8109: Hive storage plugin support reading parquet timestamp type with int64...
Thinking Chen [Wed, 2 Feb 2022 09:34:47 +0000 (17:34 +0800)] 
DRILL-8109: Hive storage plugin support reading parquet timestamp type with int64 logical type (#2431)

3 months agoDRILL-7994: Dependency version updates for severe vulnerabilities (#2432)
James Turton [Wed, 2 Feb 2022 09:18:22 +0000 (11:18 +0200)] 
DRILL-7994: Dependency version updates for severe vulnerabilities (#2432)

* Sev 8+ vuln dep updates.

* Adjust netty-tcnative version for windows-x86_64.

* Bump Avro because of severe Velocity vulns.

* Adjust expected row count to accommodate t-digest 3.3 improvements.

"The meaning of the compression parameter became a bit more strict in this
latest version. As such, the digest should be a bit smaller and tails
should be a bit more accurate." -- Ted Dunning

* Code comment whitespace formatting.

* Remove cruft from jdbc-all jar.

* Return needed Netty jars to jdbc-all.

* Adjust unit test memory allocs for TestAggregateFunctions.

* Remove references to unused artifact jersey-guava.

* Netty to 2.0.48.

* +200MiB direct memory for the unit test run.

* Upgrade jetty-server and h2 for dependabot.

* Fixes for H2 lib upgrade.

* Work around DOUBLE type DDL syntax error for pg dialect.

* Increase CI heap mem allocation for the sake of JDK 8.

* Specify parallel GC for CI to try to minimise peak mem usage.

* Cast Netty ByteBuffer to Buffer for JDK 8 compat.

* Increase CI heap mem allocation for the sake of TestAggregateFunctions.

3 months agoDRILL-8111: Remove lombok usage (#2433)
Volodymyr Vysotskyi [Wed, 2 Feb 2022 01:22:54 +0000 (03:22 +0200)] 
DRILL-8111: Remove lombok usage (#2433)

3 months agoDRILL-8115: Adds LIMIT pushdown support in the scan framework (#2441)
Paul Rogers [Tue, 1 Feb 2022 09:54:24 +0000 (01:54 -0800)] 
DRILL-8115: Adds LIMIT pushdown support in the scan framework (#2441)

* Adds LIMIT pushdown support in the scan framework

Modifies EVF and the scan framework to enforce LIMIT
pushdown across readers.

* Fix build issues

* Fixes from review comments

3 months agoDRILL-8056: Add OAuth2 Support for HTTP Rest Plugin (#2401)
Charles S. Givre [Mon, 31 Jan 2022 21:48:42 +0000 (16:48 -0500)] 
DRILL-8056: Add OAuth2 Support for HTTP Rest Plugin (#2401)

* DRILL-8056: Add OAuth2 Support for HTTP Rest Plugin

* Fixed one unit test and removed code from other branch

* Fix UT

* WIP

* Moving to Cred provider

* WIP

* UT working with Cred provider

* Successfully optained tokens

* Ready for review

* Fixed Checkstyle

* Code cleanup

* Updated docs and minor code cleanup

* Moved okhttp3 to test scope

* Fixed pom... again

* Pom fixes

* Added redirect URI to access token request

* Added token type to config

* Doc update

* Updated docs and addressed review comments

* Initial work with persistent store

* 2/3 Unit tests passing

* All unit tests passing

* Code cleanup

* Passing manual tests with Clickup and Salesforce

* Final revisions

* Fix resource leak

* Added Success FTL

* Fix resource leak

* UTs passing

* Addressed Review Comments

* Removed unused import

3 months agoDRILL-8118: Add Option to Allow Disk Use on Mongo Queries
Charles S. Givre [Mon, 31 Jan 2022 05:30:34 +0000 (00:30 -0500)] 
DRILL-8118: Add Option to Allow Disk Use on Mongo Queries

3 months agoDRILL-8116: Upgrade Apache Xerces because of CVE-2022-23437 (#2443)
kingswanwho [Sun, 30 Jan 2022 17:03:46 +0000 (01:03 +0800)] 
DRILL-8116: Upgrade Apache Xerces because of CVE-2022-23437 (#2443)

Co-authored-by: kingswanwho <jingchuanhu2017@u.northwestern.edu>
3 months agoDRILL-8118: Add Option to Allow Disk Use on Mongo Queries (#2442)
Charles S. Givre [Sun, 30 Jan 2022 11:26:39 +0000 (06:26 -0500)] 
DRILL-8118: Add Option to Allow Disk Use on Mongo Queries (#2442)

* Initial commit

* Update contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java

Co-authored-by: luoc <luoc@apache.org>
Co-authored-by: luoc <luoc@apache.org>