drill.git
6 days agoDRILL-8253: Support the limit results in kafka scan (#2580) master
luoc [Wed, 29 Jun 2022 20:05:24 +0000 (04:05 +0800)] 
DRILL-8253: Support the limit results in kafka scan (#2580)

* Addressed review comments

Co-authored-by: luoc <luocooong@qq.com>
2 weeks agoDRILL-8251: Upgrade Hadoop 2 Due To CVE (#2577)
PJ Fanning [Mon, 20 Jun 2022 13:13:50 +0000 (14:13 +0100)] 
DRILL-8251: Upgrade Hadoop 2 Due To CVE (#2577)

2 weeks agoDRILL-8240: Remove Strings.java and Add Dependency on log4j-api Instead (#2574)
PJ Fanning [Sun, 19 Jun 2022 11:46:33 +0000 (12:46 +0100)] 
DRILL-8240: Remove Strings.java and Add Dependency on log4j-api Instead (#2574)

2 weeks ago[MINOR UPDATE] Fix Port Clash Issue in HTTP Plugin Tests (#2575)
PJ Fanning [Sun, 19 Jun 2022 01:48:28 +0000 (02:48 +0100)] 
[MINOR UPDATE] Fix Port Clash Issue in HTTP Plugin Tests (#2575)

* try to fix port clash issue in http tests

* fix spelling of 'separate'

2 weeks agoissues-2552 revert netty upgrade due to test issues
PJ Fanning [Fri, 17 Jun 2022 21:36:02 +0000 (22:36 +0100)] 
issues-2552 revert netty upgrade due to test issues

2 weeks agoDRILL-8199: Convert Excel EVF1 to EVF2 (#2569)
James Turton [Fri, 17 Jun 2022 20:38:49 +0000 (22:38 +0200)] 
DRILL-8199: Convert Excel EVF1 to EVF2 (#2569)

* Upgrade Excel format plugin to EVF 2.

* Fix implict column exclusion from wildcard projection in EVF2.

2 weeks agoupdate year in copyright
PJ Fanning [Fri, 17 Jun 2022 19:35:26 +0000 (20:35 +0100)] 
update year in copyright

2 weeks agoissues-2552 netty upgrade
PJ Fanning [Fri, 17 Jun 2022 19:28:28 +0000 (20:28 +0100)] 
issues-2552 netty upgrade

3 weeks agoDRILL-8248: Fix http_request for several rows (#2573)
Vitalii Diravka [Wed, 15 Jun 2022 08:31:01 +0000 (11:31 +0300)] 
DRILL-8248: Fix http_request for several rows (#2573)

3 weeks agoDRILL-8245: Project pushdown depends on rules order and might not happen
Volodymyr Vysotskyi [Thu, 9 Jun 2022 17:11:17 +0000 (20:11 +0300)] 
DRILL-8245: Project pushdown depends on rules order and might not happen

3 weeks agoDRILL-8132: Improvement of RPC (#2464)
leon [Fri, 10 Jun 2022 04:13:51 +0000 (12:13 +0800)] 
DRILL-8132: Improvement of RPC (#2464)

Co-authored-by: feiteng.wtf <feiteng.wtf@cainiao.com>
3 weeks agoDRILL-8244: HTTP_Request Not Passing Down Config Variables (#2571)
Charles S. Givre [Thu, 9 Jun 2022 01:17:37 +0000 (21:17 -0400)] 
DRILL-8244: HTTP_Request Not Passing Down Config Variables (#2571)

* Initial Commit

* Addressed review comments, removed logging statement

4 weeks agoDRILL-8242: Fix output for HttpHelperFunctions (#2568)
Vitalii Diravka [Tue, 7 Jun 2022 15:41:41 +0000 (18:41 +0300)] 
DRILL-8242: Fix output for HttpHelperFunctions (#2568)

* DRILL-8242: Fix output for HttpHelperFunctions

- add test to check the http_get function output

* DRILL-8242: Fix output for HttpHelperFunctions

* DRILL-8242: Fix old approach for other functions

* DRILL-8242: replace String with InputStream for SimpleHTTP requests

* Fix TestHttpPlugin#testSlowResponse. Sometimes DATA_READ ERROR: Read timed out can be returned

4 weeks agoDRILL-8243: Move JSON Config Options Out of HTTP Plugin (#2570)
Charles S. Givre [Tue, 7 Jun 2022 13:53:09 +0000 (09:53 -0400)] 
DRILL-8243: Move JSON Config Options Out of HTTP Plugin (#2570)

5 weeks agoDRILL-8113: Support building with a JDK 8 target using newer JDKs (#2565)
James Turton [Wed, 1 Jun 2022 04:50:56 +0000 (06:50 +0200)] 
DRILL-8113: Support building with a JDK 8 target using newer JDKs (#2565)

Also upgrades and mostly enables the tests in TestUserBitKerberos*.java.

5 weeks agoDRILL-8149: large xlsx configs (#2483)
PJ Fanning [Sun, 29 May 2022 07:21:21 +0000 (08:21 +0100)] 
DRILL-8149: large xlsx configs (#2483)

5 weeks agoDRILL-8236: Move HttpHelperFunctions to use JSON2 reader (#2566)
Vitalii Diravka [Fri, 27 May 2022 21:05:01 +0000 (00:05 +0300)] 
DRILL-8236: Move HttpHelperFunctions to use JSON2 reader (#2566)

* DRILL-8236: Move HttpHelperFunctions to use JSON2 reader

* DRILL-8236: Move HttpHelperFunctions to use JSON2 reader

Co-authored-by: Charles Givre <cgivre@apache.org>
5 weeks agoDRILL-8232: Add support for user credentials to VaultCredentialsProvider (#2558)
James Turton [Fri, 27 May 2022 00:16:55 +0000 (02:16 +0200)] 
DRILL-8232: Add support for user credentials to VaultCredentialsProvider (#2558)

* Add support for user credentials to VaultCredentialsProvider.

* Change LOGGER var to lower case.

* Convert VaultCredentialsProvider to AppRole authentication.

BREAKING. VaultCredentialsProvider was previously configured with
a static Vault token in a BOOT option. Now it is configured with a
static AppRole and secret in BOOT options and those are used to obtain
Vault tokens.

* Fox logic and style.

* Satisfy RAT.

5 weeks agoDRILL-8237: Limit is not pushed down to scan for MSSQL (#2564)
Volodymyr Vysotskyi [Fri, 27 May 2022 00:16:33 +0000 (03:16 +0300)] 
DRILL-8237: Limit is not pushed down to scan for MSSQL (#2564)

6 weeks agoDRILL-8234: Register rules only from plugins used in the query (#2560)
Volodymyr Vysotskyi [Wed, 25 May 2022 06:38:01 +0000 (09:38 +0300)] 
DRILL-8234: Register rules only from plugins used in the query (#2560)

* DRILL-8234: Register rules only from plugins used in the query

* DRILL-8234: Obtain plugins from query context

6 weeks agoDRILL-8223: Change JDK 17 Docker base image from Oracle Linux to Debian. (#2559)
James Turton [Tue, 24 May 2022 11:14:02 +0000 (13:14 +0200)] 
DRILL-8223: Change JDK 17 Docker base image from Oracle Linux to Debian. (#2559)

6 weeks ago[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64. (#2557)
James Turton [Mon, 23 May 2022 13:10:35 +0000 (15:10 +0200)] 
[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64. (#2557)

* [MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64.

* Try using DisabledIf... instead of EnabledIf.. at class level.

* Try adding EnabledIf... annotation to static setup and teardown methods.

* Try adding a manual bypass to the @BeforeClass method.

* Try using method-level EnabledIf... annotations.

* Try to disable using os.arch = "aarch_64".

* Try using org.junit.Assume instead of annotations.

6 weeks agoDRILL-8230: Upgrade to POI 5.2.2 (#2555)
PJ Fanning [Sun, 22 May 2022 02:01:10 +0000 (03:01 +0100)] 
DRILL-8230: Upgrade to POI 5.2.2 (#2555)

6 weeks agoRevert "Fix imports" and "[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not...
James Turton [Sat, 21 May 2022 09:16:04 +0000 (11:16 +0200)] 
Revert "Fix imports" and "[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64."

Reverted commits:

 - 012d5ea2056d7caea8f3e008bb39032cafdd1c9f
 - 9c01c03367265375520a7901109d9f900c66aa25

6 weeks agoFix imports.
James Turton [Sat, 21 May 2022 08:40:49 +0000 (10:40 +0200)] 
Fix imports.

6 weeks ago[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64. 2556/head
James Turton [Sat, 21 May 2022 08:13:53 +0000 (10:13 +0200)] 
[MINOR UPDATE] Disable TestJdbcPluginWithMSSQL when not on amd64.

6 weeks agoDRILL-8229: Add Parameter to Skip Malformed Records to HTTP UDF (#2553)
Charles S. Givre [Thu, 19 May 2022 20:43:39 +0000 (16:43 -0400)] 
DRILL-8229: Add Parameter to Skip Malformed Records to HTTP UDF (#2553)

* Add Parameter to Skip Malformed Records to HTTP UDF

* Addressed review comments

6 weeks agoDRILL-8228: Drill2288GetColumnsMetadataWhenNoRowsTest regression (#2554)
Vitalii Diravka [Thu, 19 May 2022 18:21:01 +0000 (21:21 +0300)] 
DRILL-8228: Drill2288GetColumnsMetadataWhenNoRowsTest regression (#2554)

6 weeks agoDRILL-8209: Introduce rule for converting join with distinct input to semi-join ...
Volodymyr Vysotskyi [Thu, 19 May 2022 17:29:00 +0000 (20:29 +0300)] 
DRILL-8209: Introduce rule for converting join with distinct input to semi-join (#2533)

6 weeks agoDRILL-8227: JConnect and jTDS JDBC drivers do not implement Connection::getSchema...
James Turton [Thu, 19 May 2022 15:23:31 +0000 (17:23 +0200)] 
DRILL-8227: JConnect and jTDS JDBC drivers do not implement Connection::getSchema (#2551)

6 weeks agoDRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints (#2549)
Niels Basjes [Thu, 19 May 2022 00:57:08 +0000 (02:57 +0200)] 
DRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints (#2549)

* DRILL-8225: Update LogParser and Yauaa to support User-Agent Client Hints

* DRILL-8225: Replace Caffeine caching with Java 8 compliant solution

* DRILL-8225: Fix dependency issues

* DRILL-8225: Use new API for JDK8 caching

7 weeks agoDRILL-8223: Refactor auth modes dropping DRILL_PROCESS and allowing credential provid...
James Turton [Tue, 17 May 2022 18:58:05 +0000 (20:58 +0200)] 
DRILL-8223: Refactor auth modes dropping DRILL_PROCESS and allowing credential providers everywhere (#2547)

* Remove CredentialedStoragePluginConfig and drill_process auth mode.

* Add a unit test of shared_user with no creds based on H2.

* Fix compile errors.

7 weeks agoDRILL-8220: Add User Translation Support for OAuth Enabled Plugins (#2544)
Charles S. Givre [Mon, 16 May 2022 14:09:10 +0000 (10:09 -0400)] 
DRILL-8220: Add User Translation Support for OAuth Enabled Plugins (#2544)

7 weeks agoDRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (set hadoop.version...
James Turton [Mon, 16 May 2022 10:54:18 +0000 (12:54 +0200)] 
DRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (set hadoop.version) (#2550)

7 weeks agoDRILL-8224: Fix TestHttpPlugin#testSlowResponse
Vitalii Diravka [Thu, 12 May 2022 17:34:15 +0000 (20:34 +0300)] 
DRILL-8224: Fix TestHttpPlugin#testSlowResponse

7 weeks agoDRILL-8214: Replace EnumerableTableScan usage with LogicalTableScan (#2538)
Volodymyr Vysotskyi [Fri, 13 May 2022 18:06:28 +0000 (21:06 +0300)] 
DRILL-8214: Replace EnumerableTableScan usage with LogicalTableScan (#2538)

7 weeks agoDRILL-8211: Replace deprecated RelNode.getChildExps with Project.getProjects (#2535)
Volodymyr Vysotskyi [Thu, 12 May 2022 17:27:55 +0000 (20:27 +0300)] 
DRILL-8211: Replace deprecated RelNode.getChildExps with Project.getProjects (#2535)

7 weeks agoDRILL-8216: Use EVF-based JSON reader for Values operator
Volodymyr Vysotskyi [Sun, 8 May 2022 10:54:45 +0000 (13:54 +0300)] 
DRILL-8216: Use EVF-based JSON reader for Values operator

7 weeks agoDRILL-8218: Add unit tests for StorageResource REST endpoints (#2541)
James Turton [Thu, 12 May 2022 04:48:20 +0000 (06:48 +0200)] 
DRILL-8218: Add unit tests for StorageResource REST endpoints (#2541)

7 weeks agoDRILL-8222: Fix wrong func impl of concat_delim, when null-value exist in middle...
xurenhe [Wed, 11 May 2022 13:16:11 +0000 (21:16 +0800)] 
DRILL-8222: Fix wrong func impl of concat_delim, when null-value exist in middle args (#2546)

8 weeks agoDRILL-8219: Handle null catalog names returned by DB2 in storage-jdbc. (#2542)
James Turton [Tue, 10 May 2022 16:22:57 +0000 (18:22 +0200)] 
DRILL-8219: Handle null catalog names returned by DB2 in storage-jdbc. (#2542)

8 weeks agoDRILL-8210: Add substring convertlet
Volodymyr Vysotskyi [Sat, 7 May 2022 15:32:20 +0000 (18:32 +0300)] 
DRILL-8210: Add substring convertlet

8 weeks agoDRILL-8213: Replace deprecated RelNode.getRows with RelNode.estimateRowCount
Volodymyr Vysotskyi [Sat, 7 May 2022 16:29:41 +0000 (19:29 +0300)] 
DRILL-8213: Replace deprecated RelNode.getRows with RelNode.estimateRowCount

8 weeks agoDRILL-8212: Join queries fail with StackOverflowError (#2536)
Volodymyr Vysotskyi [Mon, 9 May 2022 15:52:24 +0000 (18:52 +0300)] 
DRILL-8212: Join queries fail with StackOverflowError (#2536)

8 weeks agoDRILL-8215: Remove SecurityContext from PluginConfigWrapper
Charles S. Givre [Mon, 9 May 2022 09:05:05 +0000 (05:05 -0400)] 
DRILL-8215: Remove SecurityContext from PluginConfigWrapper

8 weeks agoDRILL-8208: Create builder for SqlSelect (#2532)
Volodymyr Vysotskyi [Mon, 9 May 2022 08:46:06 +0000 (11:46 +0300)] 
DRILL-8208: Create builder for SqlSelect (#2532)

8 weeks agoDRILL-8207: Fix Username Typo and password @JsonIgnore in JDBC SerDe (#2531)
James Turton [Fri, 6 May 2022 15:42:59 +0000 (17:42 +0200)] 
DRILL-8207: Fix Username Typo and password @JsonIgnore in JDBC SerDe (#2531)

2 months agoDRILL-8207: Fix typo in JDBC config (#2530)
Charles S. Givre [Thu, 5 May 2022 21:33:40 +0000 (17:33 -0400)] 
DRILL-8207: Fix typo in JDBC config (#2530)

2 months ago[MINOR UPDATE] Update AWS Java SDK to 1.12.211
Charles S. Givre [Thu, 5 May 2022 07:13:28 +0000 (03:13 -0400)] 
[MINOR UPDATE] Update AWS Java SDK to 1.12.211

2 months agoDRILL-8204: Allow Provided Schema for HTTP Plugin in JSON Mode (#2526)
Charles S. Givre [Tue, 3 May 2022 20:32:25 +0000 (16:32 -0400)] 
DRILL-8204: Allow Provided Schema for HTTP Plugin in JSON Mode (#2526)

* Initial commit

* Map working

* WIP

* Added builder

* Lists in maps working

* Add documentation

* Cleaned up UT

* Final Revision

* Fix checkstyle

* Minor tweak

* removed extra test file

* Removed unused import

* Added inline schema support

* Addressed review comments

* Removed unused import

* Removed json string

* Final Revisions

* Fixed unit test

2 months ago[MINOR UPDATE] Fix default auth mode in AuthMode.parseOrDefault and test the case...
James Turton [Tue, 3 May 2022 14:20:33 +0000 (16:20 +0200)] 
[MINOR UPDATE] Fix default auth mode in AuthMode.parseOrDefault and test the case of JDBC. (#2528)

2 months agoDRILL-8035: Update Janino to 3.1.7 version
Volodymyr Vysotskyi [Sat, 5 Mar 2022 08:57:57 +0000 (10:57 +0200)] 
DRILL-8035: Update Janino to 3.1.7 version

2 months agoDRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (#2525)
James Turton [Sat, 30 Apr 2022 09:21:40 +0000 (11:21 +0200)] 
DRILL-8200: Update Hadoop libs to ≥ 3.2.3 for CVE-2022-26612 (#2525)

* Remove pointless Buffer casts.

Compiling Drill with JDK > 8 will still result in ByteBuffer <-> Buffer cast
exceptions at runtime when running on JDK 8 even though maven.target.version
is set to 8. Setting maven.compiler.release to 8 solves the Buffer casts
but raises a compilation error of package sun.security.jgss does not exist
for JDK 8. There were a few handwritten casts to avoid the Buffer casting
issue but many instances are not covered so the few reverted in this commit
achieve nothing.

* Update Hadoop to 3.2.3.

2 months agoDRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin (#2524)
Charles S. Givre [Thu, 28 Apr 2022 14:33:13 +0000 (10:33 -0400)] 
DRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin (#2524)

* DRILL-8202: Add Options to Skip Malformed JSON Records to HTTP Plugin

* Fixed typo

2 months agoDRILL-8195: Add Timestamp Zone offset ISO-8601 format for JSON EVF
Vitalii Diravka [Tue, 26 Apr 2022 14:09:02 +0000 (17:09 +0300)] 
DRILL-8195: Add Timestamp Zone offset ISO-8601 format for JSON EVF

2 months agoDRILL-8037: Add V2 JSON Format Plugin based on EVF
Vitalii Diravka [Fri, 5 Nov 2021 02:21:40 +0000 (04:21 +0200)] 
DRILL-8037: Add V2 JSON Format Plugin based on EVF

* Enable store.json.enable_v2_reader by default
* Fix TestJsonReader doulbe quotes test cases. Update jackson 2.12.1 -> 2.13.0
* Disable V2 for experimental UNION datatype
* Fix regressions
* Fix json Schema Provision (it wasn't provided for JsonLoaderBuilder). The previous schema provision was a fake, the reader schema was infered from the json content. It fixes the scan and reader schema validation. And it starts to apply the provided schema to ANALYZE COMMANDS, fixed TestMetastoreWithEasyFormatPlugin#testAnalyzeOnJsonTable

2 months agoDRILL-6953: Merge row set-based JSON reader
Paul Rogers [Tue, 14 Apr 2020 02:45:16 +0000 (19:45 -0700)] 
DRILL-6953: Merge row set-based JSON reader

* Snapshot: reworked JSON field parser creation
* Updated JSON loader
* Redo value listener with tokens
* Extended long type works
* More simple extended types
* Added $date
* Binary type
* All extended type except arrays
* Extended arrays partly working
* More arrays work
* Refactor element parser interfaces
* Rename RowSetTests --> RowSetTest
* More factory cleanup
* Revised unknown field creation
* In middle of factory/parser restructuring
* Scalars, object, some variants work again
* JSON loader tests pass
* File cleanup
* Old extended types test passes
* Renamed JSON packages
* Tested extended provided types

2 months agoDRILL-8155: Introduce New Plugin Authentication Modes (#2516)
Charles S. Givre [Wed, 27 Apr 2022 00:15:37 +0000 (20:15 -0400)] 
DRILL-8155: Introduce New Plugin Authentication Modes (#2516)

* Do not set the read-only hint on JDBC connections.

* Outline of different auth modes in storage-jdbc.

In this commit, a new `authMode` storage config supporting three new auth modes
is defined: shared user (default), user translation (user is translated to some
other user from the external storage) and impersonation (the external storage
and JDBC driver provide support for impersonating the Drill query user).

The JdbcStoragePlugin is enhanced to be able to work with a lookup table of
connection pools, where a pool is dedicated to each query user except when
in shared user auth mode.

Planning and execution time APIs are also enhanced to transmit a user
credentials object for the query user, instead of just its username.
This allows for the expansion of the UserCredentials protobuf type to
include some optional extra credentials, e.g. in an array of byte arrays.
These credentials may be relevant in the user tranlsation mode when a
credential provider must be accessed in order to obtain the creds to be used
for the external system.

* Rebased to current master and build fixes

* Build works, cred stuff added

* Credentials being saved and pushed down to storage plugin

* UI now closing properly

* User Translation working for HTTP plugin

* HTTP unit tests passing

* WIP

* Fixed import

* User Credentials now being stored in credential provider

* Working

* Fixed TPCH Unit Tests

* Fix CredProvider SerDe Test

* Added unit tests for JDBC

* Code cleanup

* Fix LGTM alerts

* Correct username now populating Group Scan

* Username to Subscan

* Remove PerUserUsernamePasswordCredentials class.

* Remove getUserCredentials from CredentialsProvider.

* Planning errors fixed

* Removed unused imports

* Fixed minor issues

* Unit test fixes

* WIP.

* Fix CodeQL Alert

* Ignore LGTM False Positive

* Fix tainted string LGTM alert

* Revert LGTM Comment

* Addressed review comments

* Use fixed size Guava caches in JDBC convetion and dialect factories.

These replace Maps with no size limit that might have grown without
bound. LRU eviction begins when the cache size limit is reached.

* Add a TTL to the JDBC dialect and convention caches.

Co-authored-by: James Turton <james@somecomputer.xyz>
2 months agoDRILL-8194: Fix the function of REPLACE throws IndexOutOfBoundsException If text...
xurenhe [Sun, 24 Apr 2022 02:44:10 +0000 (10:44 +0800)] 
DRILL-8194: Fix the function of REPLACE throws IndexOutOfBoundsException If text's length is more than previously applied (#2522)

2 months agoDRILL-8013: Drill attempts to push "$SUM0" to JDBC storage plugin for AVG (#2521)
Volodymyr Vysotskyi [Wed, 20 Apr 2022 16:27:14 +0000 (19:27 +0300)] 
DRILL-8013: Drill attempts to push "$SUM0" to JDBC storage plugin for AVG (#2521)

2 months agoDRILL-8193: Incorrect Annotation used for HttpJsonOptions (#2519)
Charles S. Givre [Wed, 13 Apr 2022 04:01:37 +0000 (00:01 -0400)] 
DRILL-8193: Incorrect Annotation used for HttpJsonOptions (#2519)

2 months agoDRILL-8192: Cassandra queries fail when enabled Mongo plugin (#2518)
Volodymyr Vysotskyi [Mon, 11 Apr 2022 00:29:03 +0000 (03:29 +0300)] 
DRILL-8192: Cassandra queries fail when enabled Mongo plugin (#2518)

2 months agoDRILL-8191: HTTP Request Function Not Detecting JSON Config (#2517)
Charles S. Givre [Thu, 7 Apr 2022 18:10:37 +0000 (14:10 -0400)] 
DRILL-8191: HTTP Request Function Not Detecting JSON Config (#2517)

3 months agoDRILL-8174: Convert Avro format to EVF2 (#2511)
luoc [Mon, 4 Apr 2022 16:21:25 +0000 (00:21 +0800)] 
DRILL-8174: Convert Avro format to EVF2 (#2511)

* DRILL-8174: Convert Avro format to EVF2

* Addressed review comments

3 months agoDRILL-8186: Convert XML format to EVF2 (#2512)
luoc [Sun, 3 Apr 2022 21:46:02 +0000 (05:46 +0800)] 
DRILL-8186: Convert XML format to EVF2 (#2512)

3 months agoDRILL-8187: Dialect factory returns ANSI SQL dialect for BigQuery (#2513)
Volodymyr Vysotskyi [Sun, 3 Apr 2022 21:45:50 +0000 (00:45 +0300)] 
DRILL-8187: Dialect factory returns ANSI SQL dialect for BigQuery (#2513)

3 months agoFix projection of map arrays and nested maps (#2510)
Paul Rogers [Sun, 3 Apr 2022 08:42:27 +0000 (01:42 -0700)] 
Fix projection of map arrays and nested maps (#2510)

Fixes the rather complex area in EVF2 that handles
projection of nested maps and map arrays to properly pass
along the offset vector for map arrays, and set the value
count for nested maps.

3 months agoDRILL-8181: Accept nullable args in HTTP plugin UDFs, fix HikariCP default parm names...
James Turton [Thu, 31 Mar 2022 13:49:49 +0000 (15:49 +0200)] 
DRILL-8181: Accept nullable args in HTTP plugin UDFs, fix HikariCP default parm names (#2509)

3 months agoDRILL-8176: minor issue in previous jackson bom (#2508)
PJ Fanning [Thu, 31 Mar 2022 12:21:24 +0000 (14:21 +0200)] 
DRILL-8176: minor issue in previous jackson bom (#2508)

* minor issue in previous jackson bom

* Update pom.xml

3 months agoDRILL-8177: Bump default TLS version to 1.3 (#2505)
Rymar Maksym [Mon, 28 Mar 2022 18:53:38 +0000 (21:53 +0300)] 
DRILL-8177: Bump default TLS version to 1.3 (#2505)

* Bump default TLS version to 1.3. Fix web server log

* Change TLS version in tests. Change value of TLS constants

3 months agoDRILL-8175: Update Drill release script after 1.20 (#2503)
James Turton [Sun, 27 Mar 2022 11:20:27 +0000 (13:20 +0200)] 
DRILL-8175: Update Drill release script after 1.20 (#2503)

* Set DRILL_PID_DIR in Dockerfile to writable location for distributed mode.

Some users of the images built from this Dockerfile customise
them so that they launch Drill in distributed mode instead of
embedded mode.  This change saves them from having to set
DRILL_PID_DIR themselves in order to succeed.

* Update release script and instructions after the release of 1.20.

- Add support for specifying a build profile such as "hadoop-2".
- Update instuctions for the Drill web site.
- Update instructions for uploading RCs (no more home.apache)
- Some fixes.

3 months agoDRILL-8178: Bump AWS Libraries to Latest Version (#2506)
Charles S. Givre [Sun, 27 Mar 2022 08:16:51 +0000 (04:16 -0400)] 
DRILL-8178: Bump AWS Libraries to Latest Version (#2506)

3 months agoDRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613 (#2493)
luoc [Fri, 25 Mar 2022 18:28:20 +0000 (02:28 +0800)] 
DRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613 (#2493)

* DRILL-8164: Upgrade metadata-extractor because of CVE-2022-24613

* Update the ProfileCopyright tag name

* Update the mov format name

* Add the QuickTime.Rotation tag

* Bump metadata-extractor to 2.17.0

3 months agoDRILL-8176: Upgrade Jackson Due to CVE-2020-36518 (#2504)
PJ Fanning [Fri, 25 Mar 2022 18:26:40 +0000 (19:26 +0100)] 
DRILL-8176: Upgrade Jackson Due to CVE-2020-36518 (#2504)

3 months ago[MINOR UPDATE] Fix the word spelling error (#2502)
xzh [Fri, 25 Mar 2022 10:27:38 +0000 (18:27 +0800)] 
[MINOR UPDATE] Fix the word spelling error (#2502)

3 months ago[MINOR UPDATE] Add license to CodeQL YAML (#2501)
luoc [Fri, 25 Mar 2022 07:07:17 +0000 (15:07 +0800)] 
[MINOR UPDATE] Add license to CodeQL YAML (#2501)

3 months agoDRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2 (#2485)
Paul Rogers [Thu, 24 Mar 2022 14:52:19 +0000 (07:52 -0700)] 
DRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2 (#2485)

* DRILL-8086: Convert the CSV (AKA "compliant text") reader to EVF V2

Also includes:

* DRILL-8159: Convert the HTTPD reader to use EVF V2

* Build fix

* Changes from review comments

* Fix test issue

3 months agoUpdate codeql-analysis.yml
Charles S. Givre [Thu, 24 Mar 2022 12:30:10 +0000 (07:30 -0500)] 
Update codeql-analysis.yml

Removed cpp from code analysis

3 months agoCreate codeql-analysis.yml
Charles S. Givre [Wed, 23 Mar 2022 22:01:57 +0000 (17:01 -0500)] 
Create codeql-analysis.yml

3 months agoDRILL-8171: Convert SequenceFile to EVF2 (#2498)
luoc [Tue, 22 Mar 2022 15:06:10 +0000 (23:06 +0800)] 
DRILL-8171: Convert SequenceFile to EVF2 (#2498)

3 months agoDRILL-8165: Upgrade liquibase because of CVE-2022-0839 (#2497)
luoc [Mon, 21 Mar 2022 14:09:59 +0000 (22:09 +0800)] 
DRILL-8165: Upgrade liquibase because of CVE-2022-0839 (#2497)

3 months agoDRILL-8172: Use the specified memory usage for Travis CI (#2500)
luoc [Mon, 21 Mar 2022 08:30:57 +0000 (16:30 +0800)] 
DRILL-8172: Use the specified memory usage for Travis CI (#2500)

3 months agoDRILL-8169: Add UDFs to HTTP Plugin to Facilitate Joins (#2496)
Charles S. Givre [Fri, 18 Mar 2022 17:12:28 +0000 (13:12 -0400)] 
DRILL-8169: Add UDFs to HTTP Plugin to Facilitate Joins (#2496)

* Initial commit

* Unit tests wroking

* Added configs from constants

* Working with Storage plugins

* Removed cache files

* Fixed cache directory

* Updated docs

* Removed unneeded pom.xml modifications

* Addressed Review comments

* Converted holder to VarCharHolder[]

3 months agoIncrement the Drill version number to 2.0.0-SNAPSHOT. (#2490)
James Turton [Fri, 18 Mar 2022 16:36:11 +0000 (18:36 +0200)] 
Increment the Drill version number to 2.0.0-SNAPSHOT. (#2490)

3 months agoDRILL-8168: Do not duplicate attempts to impersonate a user in the REST API (#2495)
James Turton [Wed, 16 Mar 2022 10:12:05 +0000 (12:12 +0200)] 
DRILL-8168: Do not duplicate attempts to impersonate a user in the REST API (#2495)

3 months agoDRILL-8167: Add JSON Config Options to Format Config (#2494)
Charles S. Givre [Tue, 15 Mar 2022 12:21:37 +0000 (08:21 -0400)] 
DRILL-8167: Add JSON Config Options to Format Config (#2494)

* Initial work'

* Initial commit

* Added unit tests

* Code cleanup and final param

* Fixed Unit Test

* Addressed Review Comments

3 months agoDRILL-8156: Declare and chown a /data VOLUME in the Drill Dockerfile (#2491)
James Turton [Sun, 13 Mar 2022 15:26:51 +0000 (17:26 +0200)] 
DRILL-8156: Declare and chown a /data VOLUME in the Drill Dockerfile (#2491)

* Add a mountpoint and VOLUME for local storage to Dockerfile.

* Address review comments concerning layer ordering.

* Fix image size blowup by moving chmod to intermediate container.

* Combine RUN commands in Dockerfile.

3 months agoDRILL-8162: Add OpenAPI Specification documentation for Drill's REST API (#2489)
estherbuchwalter [Fri, 11 Mar 2022 21:23:33 +0000 (16:23 -0500)] 
DRILL-8162: Add OpenAPI Specification documentation for Drill's REST API (#2489)

* Excluded swagger dependencies to fix enforcer error

* Added swagger core dependencies and swagger version

* Tested adding swagger annotations

* Added external docs in the openAPI Specification/swagger annotations

* Reverted code reformat. Deencoded URLs. Reformatted log messages.

* More format changes.

3 months agoDRILL-8161: Add Global Credentials to HTTP Storage Plugin (#2486)
Charles S. Givre [Wed, 9 Mar 2022 22:43:08 +0000 (17:43 -0500)] 
DRILL-8161: Add Global Credentials to HTTP Storage Plugin (#2486)

* DRILL-8161: Add Global Credentials to HTTP Storage Plugin

* Removed unused import

* Formatting from code review

* Updated README

3 months ago[MINOR UPDATE] Fix license for Stalebot Config (#2488)
Volodymyr Vysotskyi [Tue, 8 Mar 2022 15:06:54 +0000 (17:06 +0200)] 
[MINOR UPDATE] Fix license for Stalebot Config (#2488)

3 months ago[MINOR UPDATE] Add Stalebot Config (#2487)
Charles S. Givre [Tue, 8 Mar 2022 05:26:53 +0000 (00:26 -0500)] 
[MINOR UPDATE] Add Stalebot Config (#2487)

4 months agoDRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case ...
Vitalii Diravka [Sun, 6 Mar 2022 16:02:12 +0000 (18:02 +0200)] 
DRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case (#2471)

* Set Brotli codec jar and test to occur only on Linux amd64.

* DRILL-8145: Fix flaky TestDrillbitResilience#memoryLeaksWhenCancelled test case

- changing timeout for TestDrillbitResilience tests
- timing tuning for memoryLeaksWhenCancelled
- update TestContainers version
- -DforkCount=1 for Travis Maven build
- directMemoryMb: 2500 -> 4500 leads to less occasinal test failures

Co-authored-by: James Turton <james@somecomputer.xyz>
4 months agoDRILL-8154: Upgrade to poi 5.2.1 (#2480)
PJ Fanning [Sat, 5 Mar 2022 12:49:43 +0000 (13:49 +0100)] 
DRILL-8154: Upgrade to poi 5.2.1 (#2480)

4 months agoDRILL-8140: Add JSON Post Body to HTTP Rest Storage Plugin (#2474)
Charles S. Givre [Thu, 3 Mar 2022 18:24:18 +0000 (13:24 -0500)] 
DRILL-8140: Add JSON Post Body to HTTP Rest Storage Plugin (#2474)

* Initial commit

* Addressed Review Comments

* Updated Docs

* Code cleanup per code review

4 months agoDRILL-8151: Add support for more ElasticSearch and Cassandra data types (#2477)
Volodymyr Vysotskyi [Thu, 3 Mar 2022 05:00:23 +0000 (07:00 +0200)] 
DRILL-8151: Add support for more ElasticSearch and Cassandra data types (#2477)

4 months agoDRILL-8153: Convert OAuth REST APIs to JSON #2479
Charles S. Givre [Wed, 2 Mar 2022 23:27:40 +0000 (18:27 -0500)] 
DRILL-8153: Convert OAuth REST APIs to JSON #2479

4 months ago[DRILL-8150] log4j 2.17.2 in format-excel (#2476)
PJ Fanning [Tue, 1 Mar 2022 14:21:07 +0000 (15:21 +0100)] 
[DRILL-8150] log4j 2.17.2 in format-excel (#2476)

4 months agoDRILL-8085: EVF V2 support in the "Easy" format plugin (#2419)
Paul Rogers [Tue, 1 Mar 2022 08:35:57 +0000 (00:35 -0800)] 
DRILL-8085: EVF V2 support in the "Easy" format plugin (#2419)

4 months agoDRILL-8148: Add REST Endpoints to Update OAuth Tokens (#2473)
Charles S. Givre [Mon, 28 Feb 2022 17:45:34 +0000 (12:45 -0500)] 
DRILL-8148: Add REST Endpoints to Update OAuth Tokens (#2473)

* Initial commit

* Added unit tests

* Fixed unused imports

* Addressed Review comments

* Fixed Link Spelling

4 months agoAdd jackson-bom (#2454)
Rymar Maksym [Sun, 27 Feb 2022 00:36:22 +0000 (02:36 +0200)] 
Add jackson-bom (#2454)