3 years ago[maven-release-plugin] prepare release drill-1.15.0 1.15.0 drill-1.15.0
Vitalii Diravka [Mon, 24 Dec 2018 18:09:05 +0000 (20:09 +0200)] 
[maven-release-plugin] prepare release drill-1.15.0

3 years agoDRILL-6920: Fix TestClient.testBasics() yarn test failure
Volodymyr Vysotskyi [Fri, 21 Dec 2018 12:37:45 +0000 (14:37 +0200)] 
DRILL-6920: Fix TestClient.testBasics() yarn test failure

closes #1582

3 years agoDRILL-6922: Do not set return result set option on query level if it is the same...
Arina Ielchiieva [Sun, 23 Dec 2018 15:10:39 +0000 (17:10 +0200)] 
DRILL-6922: Do not set return result set option on query level if it is the same as current value

1. Rename return result option name to `exec.query.return_result_set_for_ddl`.
2. Add check if option value is the same as current value in DrillSqlWorker before setting result set option on query level.
3. Separate Session and Query options on Web UI.

closes #1584

3 years agoDRILL-6925: Unable to generate Protobuf
Vitalii Diravka [Sun, 23 Dec 2018 20:51:15 +0000 (22:51 +0200)] 
DRILL-6925: Unable to generate Protobuf

- use ${maven.multiModuleProjectDirectory}/header to find header file from any submodule
- suppress UnresolvedMavenProperty, since IDE expects that property should be set explicitly
- update "kr.motd.maven:os-maven-plugin" github.com/trustin/os-maven-plugin to the latest 1.6.1 version
- correction of ${user.name} propery for "maven-jar-plugin" <Built-By>
- update "apache-rat-plugin" to solve undefined "excludeSubprojects" in IDE
- regenerate Java and C++ protobuf files

closes #1585

3 years agoDRILL-6919: Fix compilation error in TestGracefulShutdown class for mapr profile
Volodymyr Vysotskyi [Thu, 20 Dec 2018 21:39:39 +0000 (23:39 +0200)] 
DRILL-6919: Fix compilation error in TestGracefulShutdown class for mapr profile

3 years agoDRILL-6915: Disable generation of test tables with case-sensitive names for non-Linux...
Volodymyr Vysotskyi [Thu, 20 Dec 2018 12:22:21 +0000 (14:22 +0200)] 
DRILL-6915: Disable generation of test tables with case-sensitive names for non-Linux systems

closes #1580

3 years agoDRILL-6916: Fix extraneous "${project.basedir}/src/site/resources/repo/" directory...
Vitalii Diravka [Thu, 20 Dec 2018 10:21:18 +0000 (12:21 +0200)] 
DRILL-6916: Fix extraneous "${project.basedir}/src/site/resources/repo/" directory appearance

- update HBase lib from 2.1.0 to 2.1.1 version (with fix for HBASE-21005)

closes #1579

3 years agoDRILL-6912: NPE when other drillbit is already running
Igor Guzenko [Wed, 19 Dec 2018 11:45:02 +0000 (13:45 +0200)] 
DRILL-6912: NPE when other drillbit is already running

closes #1577

3 years agoDRILL-6913: Fix multiple error output in the console
Arina Ielchiieva [Wed, 19 Dec 2018 13:01:59 +0000 (15:01 +0200)] 
DRILL-6913: Fix multiple error output in the console

1. Bump up Calcite version to 1.17.0-drill-r2 (includes CALCITE-2463).
2. Optimized exception handling in DrillSqlWorker getPlan method.

closes #1578

3 years agoDRILL-6900: maven-release-plugin failure on the release:perform stage
Vitalii Diravka [Thu, 13 Dec 2018 14:17:55 +0000 (16:17 +0200)] 
DRILL-6900: maven-release-plugin failure on the release:perform stage

- downgrade maven-javadoc-plugin version
- update some Drill maven plugins versions and move them to pluginManagement block
- bump up lowest maven version supported by Drill in correspondence to org.apache.maven dependencies

closes #1574

3 years agoDRILL-6895: Warnings for license headers checks and wrong license headers in some...
Vitalii Diravka [Tue, 11 Dec 2018 12:30:47 +0000 (14:30 +0200)] 
DRILL-6895: Warnings for license headers checks and wrong license headers in some files

- update license-maven-plugin plugin custom mappings
- update files licenses headers
- update from "/usr/bin/env bash" to "/bin/bash" shebang line in shell script files

3 years agoDRILL-6878: Use DrillPushRowKeyJoinToScan rule on DrillJoin pattern to account for...
Gautam Parai [Tue, 4 Dec 2018 02:03:23 +0000 (18:03 -0800)] 
DRILL-6878: Use DrillPushRowKeyJoinToScan rule on DrillJoin pattern to account for DrillSemiJoin

closes #1568

3 years agoDRILL-6889: Exclude Drill unit tests to avoid Travis timing out
Vitalii Diravka [Fri, 7 Dec 2018 17:44:47 +0000 (19:44 +0200)] 
DRILL-6889: Exclude Drill unit tests to avoid Travis timing out

closes #1567

3 years agoDRILL-6877: NPE when starting Drillbit
Venkata Jyothsna Donapati [Mon, 3 Dec 2018 18:19:06 +0000 (10:19 -0800)] 
DRILL-6877: NPE when starting Drillbit

closes #1560

3 years agoDRILL-6791: Scan projection framework
Paul Rogers [Fri, 12 Oct 2018 05:47:49 +0000 (22:47 -0700)] 
DRILL-6791: Scan projection framework

The "schema projection" mechanism:

* Handles none (SELECT COUNT\(*)), some (SELECT a, b, x) and all (SELECT *) projection.
* Handles null columns (for projection a column "x" that does not exist in the base table.)
* Handles constant columns as used for file metadata (AKA "implicit" columns).
* Handle schema persistence: the need to reuse the same vectors across different scanners
* Provides a framework for consuming externally-supplied metadata
* Since we don't yet have a way to provide "real" metadata, obtains metadata hints from
  previous batches and from the projection list (a.b implies that "a" is a map, c[0]
  implies that "c" is an array, etc.)
* Handles merging the set of data source columns and null columns to create the final output batch.
* Running tests found a failure due to an uninialized "bits" vector. Added code to explicitly fill
  the bits vectors with zeros in the "result set loader."

3 years agoDRILL-6886: Cancel Query Button for running queries
Kunal Khatua [Mon, 10 Dec 2018 06:40:19 +0000 (22:40 -0800)] 
DRILL-6886: Cancel Query Button for running queries

The Freemarker template missed a check for a query's Running state to display the [Cancel] button.

The fix is trivial and only needs the check.

3 years agoDRILL-6887: Fix FunctionInitializerTest.init() failure
Vitalii Diravka [Fri, 7 Dec 2018 13:55:48 +0000 (15:55 +0200)] 
DRILL-6887: Fix FunctionInitializerTest.init() failure

close apache/drill#1566

3 years agoDRILL-6874: Close input stream after AsyncPageReaderTask is completed
Bohdan Kazydub [Thu, 6 Dec 2018 13:03:25 +0000 (15:03 +0200)] 
DRILL-6874: Close input stream after AsyncPageReaderTask is completed

close apache/drill#1565

3 years agoDRILL-6883: Force reload of options after resetting parameter
Kunal Khatua [Thu, 6 Dec 2018 07:34:26 +0000 (23:34 -0800)] 
DRILL-6883: Force reload of options after resetting parameter

If an update is initiated, the webpage loaded changes. Attempting a reset tries to reload this page, but it does not show the updated values from the back end.
This patch forces the reload of updated values by redirecting to /options page

Use AJAX call for Update and Reset

Eliminates the need for window.location=<redirectURL>

is Number check

Added comments to explain extraction

close apache/drill#1563

3 years agoDRILL-6751: Upgrade Apache parent POM to version 21
Vitalii Diravka [Fri, 16 Nov 2018 01:40:21 +0000 (17:40 -0800)] 
DRILL-6751: Upgrade Apache parent POM to version 21

- Update apache.pom file version to 21 (with updating some maven plugins versions)
- Include Drill's sources jars on assembly stage in <moduleSets> (not <dependencySets>)
  for properincluding jars with last apache-21.pom
- Separate "distro-assembly" to the two execution stages to avoid:
  [WARNING] Assembly file: <DRILL_HOME>/distribution/target/apache-drill-1.15.0-SNAPSHOT is not a regular
  file (it may be a directory). It cannot be attached to the project build for installation or deployment.
- Remove unsused <include>/<exclude> in assebly descriptor to avoid:
  [WARNING] The following patterns were never triggered in this artifact inclusion filter
- Update "maven-assembly-plugin" version
- Update "slf4j" version
- Update "mockito-core" version
- Update "bcpkix-jdk15on" (Bouncy Castle Cryptography APIs) version

close apache/drill#1561

3 years agoDRILL-4546: Only generate one zip archive when using apache-release profile
Laurent Goujon [Mon, 28 Mar 2016 20:31:53 +0000 (13:31 -0700)] 
DRILL-4546: Only generate one zip archive when using apache-release profile

Drill root pom doesn't override completely Apache parent pom configuration
regarding assemblies, which caused a zip archive of the project to be generated
twice, and deployed to a remote server twice too.

The fix uses the plugin properties to override the configuration. Also remove
Drill source assembly descriptor as the Apache parent project provides the same one.

3 years agoDRILL-6876: Enable CircleCI builds for JDK 9-11
Volodymyr Vysotskyi [Fri, 16 Nov 2018 13:43:27 +0000 (15:43 +0200)] 
DRILL-6876: Enable CircleCI builds for JDK 9-11

close apache/drill#1558

3 years agoDRILL-6882: Handle the cases where RowKeyJoin's left pipeline being called multiple...
Hanumath Rao Maduri [Sat, 30 Sep 2017 20:26:40 +0000 (13:26 -0700)] 
DRILL-6882: Handle the cases where RowKeyJoin's left pipeline being called multiple times.

close apache/drill#1562

3 years agoAdd Vitalii's GPG key
Vitalii Diravka [Wed, 5 Dec 2018 13:54:01 +0000 (15:54 +0200)] 
Add Vitalii's GPG key

3 years agoDRILL-6866: Upgrade to SqlLine 1.6.0
Arina Ielchiieva [Thu, 22 Nov 2018 15:27:15 +0000 (15:27 +0000)] 
DRILL-6866: Upgrade to SqlLine 1.6.0

1. Changed SqlLine version to 1.6.0.
2. Overridden new getVersion method in DrillSqlLineApplication.
3. Set maxColumnWidth to 80 to avoid issue described in DRILL-6769.
4. Changed colorScheme to obsidian.
5. Output null value for varchar / char / boolean types as null instead of empty string.
6. Changed access modifier from package default to public for JDBC classes that implement external interfaces to avoid issues when calling methods from these classes using reflection.

closes #1556

3 years agoDRILL-6867: WebUI Query editor cursor position
ozinoviev [Fri, 23 Nov 2018 09:34:36 +0000 (12:34 +0300)] 
DRILL-6867: WebUI Query editor cursor position

closes #1551

3 years agoDRILL-6863: Drop table is not working if path within workspace starts with "/"
Bohdan Kazydub [Wed, 28 Nov 2018 14:51:18 +0000 (16:51 +0200)] 
DRILL-6863: Drop table is not working if path within workspace starts with "/"

- Made workspace to be honored when table/view name starts with "/" for DROP TABLE, DROP VIEW, CREATE VIEW and SELECT from view queries;
- Made "/{name}" and "{name}" to be equivalent names (the leading "/" is removed) when creating temporary tables so that SELECT ... FROM "/{name}" ... and SELECT ... FROM "{name}" ... produce the same results and behave as regular tables in the context.

closes #1557

3 years agoDRILL-6792: Find the right probe side fragment wrapper & fix DrillBuf reference count...
weijie.tong [Sun, 14 Oct 2018 11:41:51 +0000 (19:41 +0800)] 
DRILL-6792: Find the right probe side fragment wrapper & fix DrillBuf reference count bugs & tune the execution flow & support left deep tree

closes #1504

3 years agoDRILL-6039: Fixed drillbit.sh script to do graceful shutdown
dvjyothsna [Fri, 9 Nov 2018 21:56:39 +0000 (13:56 -0800)] 
DRILL-6039: Fixed drillbit.sh script to do graceful shutdown

closes #1536

3 years agoDRILL-6864: Upgrade the git-commit-id plugin to 2.2.5
Ben-Zvi [Tue, 27 Nov 2018 03:06:52 +0000 (19:06 -0800)] 
DRILL-6864: Upgrade the git-commit-id plugin to 2.2.5

3 years agoDRILL-6834: Introduce option to disable result set for DDL queries for JDBC connection
Bohdan Kazydub [Wed, 21 Nov 2018 01:56:49 +0000 (03:56 +0200)] 
DRILL-6834: Introduce option to disable result set for DDL queries for JDBC connection

- Added session-scoped option `drill.exec.fetch_resultset_for_ddl` to control whether update count or result set should be returned for JDBC connection session. By default the option is set to `true` which ensures that result set is returned;
- Updated Drill JDBC: `DrillCursor` and `DrillStatement` to achieve desired behaviour.

closes #1549

3 years agoDRILL-6865: Filter is not removed from the plan when parquet table fully matches...
Volodymyr Vysotskyi [Fri, 23 Nov 2018 09:34:13 +0000 (11:34 +0200)] 
DRILL-6865: Filter is not removed from the plan when parquet table fully matches the filter

closes #1552

3 years agoDRILL-6865: Query returns wrong result when filter pruning happens
Volodymyr Vysotskyi [Wed, 21 Nov 2018 12:08:17 +0000 (14:08 +0200)] 
DRILL-6865: Query returns wrong result when filter pruning happens

3 years agoDRILL-6870: Upgrade to ANTLR4
Arina Ielchiieva [Tue, 20 Nov 2018 12:27:55 +0000 (14:27 +0200)] 
DRILL-6870: Upgrade to ANTLR4

closes #1554

3 years agoDRILL-6868: Upgrade Janino compiler to 3.0.11
Volodymyr Vysotskyi [Fri, 19 Oct 2018 15:04:12 +0000 (18:04 +0300)] 
DRILL-6868: Upgrade Janino compiler to 3.0.11

- Remove workaround where removing adjacent ALOAD-POP instruction pairs
- Remove ModifiedUnparser and use DeepCopier for modifying methods instead of modifying it with custom Unparser implementation

closes #1553

3 years agoDRILL-6858: Add functionality to list directories / files with exceptions suppression
Arina Ielchiieva [Tue, 20 Nov 2018 15:06:07 +0000 (17:06 +0200)] 
DRILL-6858: Add functionality to list directories / files with exceptions suppression

1. Add listDirectoriesSafe, listFilesSafe, listAllSafe in FileSystemUtil and DrillFileSystemUtil classes.
2. Use FileSystemUtil.listAllSafe during listing files in show files command and information_schema.files table.

closes #1547

3 years agoDRILL-6857: Read only required row groups in a file when limit push down is applied
Arina Ielchiieva [Tue, 20 Nov 2018 18:08:38 +0000 (20:08 +0200)] 
DRILL-6857: Read only required row groups in a file when limit push down is applied

closes #1548

3 years agoDRILL-6850: Allow configuring table names case sensitivity for JDBC storage plugin
Volodymyr Vysotskyi [Thu, 15 Nov 2018 11:07:44 +0000 (13:07 +0200)] 
DRILL-6850: Allow configuring table names case sensitivity for JDBC storage plugin

closes #1542

3 years agoDRILL-6850: Force setting DRILL_LOGICAL Convention for DrillRelFactories and DrillFil...
Volodymyr Vysotskyi [Tue, 13 Nov 2018 13:21:39 +0000 (15:21 +0200)] 
DRILL-6850: Force setting DRILL_LOGICAL Convention for DrillRelFactories and DrillFilterRel

- Fix workspace case insensitivity for JDBC storage plugin

3 years agoDRILL-6850: JDBC integration tests failures
Vitalii Diravka [Fri, 2 Nov 2018 13:41:37 +0000 (15:41 +0200)] 
DRILL-6850: JDBC integration tests failures

- Fix RDBMS integration tests (expected decimal output and testCrossSourceMultiFragmentJoin)
- Update libraries versions
- Resolve NPE for empty result

3 years agoDRILL-6847: Add Query Metadata to RESTful Interface
Charles S. Givre [Wed, 21 Nov 2018 13:15:00 +0000 (08:15 -0500)] 
DRILL-6847: Add Query Metadata to RESTful Interface

closes #1539

3 years agoDRILL-6818: Add descriptions to secondary index options.
Aman Sinha [Mon, 19 Nov 2018 19:25:59 +0000 (11:25 -0800)] 
DRILL-6818: Add descriptions to secondary index options.

closes #1545

3 years agoDRILL-6691: Unify checkstyle-config.xml files.
Hanumath Rao Maduri [Wed, 31 Oct 2018 21:39:25 +0000 (14:39 -0700)] 
DRILL-6691: Unify checkstyle-config.xml files.

closes #1550

3 years agoDRILL-6668: In Web UI, highlight options that are not default values
Kunal Khatua [Thu, 15 Nov 2018 06:28:48 +0000 (22:28 -0800)] 
DRILL-6668: In Web UI, highlight options that are not default values

This commit introduces a new button on the options page that allows a user to reset an option to its system default value.
To simplify things, a tooltip is shown when the mouse hovers over the button. If the option value is already default, the button is disabled.

Currently, the Update button redirects to /option/optionName . This change reuses what we already are using to set the default (using AJAX) and auto-refreshing
Switch [Default] label to [Reset]
Patch To Pass StatusResourcesTest

closes #1543

3 years agoDRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM
ozinoviev [Mon, 27 Aug 2018 12:48:52 +0000 (15:48 +0300)] 
DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

closes #1446

3 years agoDRILL-6861: Hash-Join should not exit after an empty probe-side spilled partition
Ben-Zvi [Tue, 20 Nov 2018 02:55:45 +0000 (18:55 -0800)] 
DRILL-6861: Hash-Join should not exit after an empty probe-side spilled partition

3 years agoDRILL-6844: Query with ORDER BY DESC on indexed column does not pick secondary index.
Hanumath Rao Maduri [Fri, 19 Oct 2018 02:35:55 +0000 (19:35 -0700)] 
DRILL-6844: Query with ORDER BY DESC on indexed column does not pick secondary index.

3 years agoDRILL-6853: Make the complex parquet reader batch max row size configurable
Salim Achouche [Thu, 15 Nov 2018 22:40:48 +0000 (14:40 -0800)] 
DRILL-6853: Make the complex parquet reader batch max row size configurable

3 years agoDRILL-6833: Support for pushdown of rowkey based joins
Gautam Parai [Wed, 3 Jan 2018 19:06:41 +0000 (11:06 -0800)] 
DRILL-6833: Support for pushdown of rowkey based joins

closes #1532

3 years agoDRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs
Gautam Parai [Wed, 3 Oct 2018 23:21:04 +0000 (16:21 -0700)] 
DRILL-6770: JsonTableGroupScan should use new MapRDB 6.1.0 APIs

closes #1489

3 years agoDRILL-6744: Support varchar and decimal push down
Arina Ielchiieva [Wed, 31 Oct 2018 18:24:03 +0000 (20:24 +0200)] 
DRILL-6744: Support varchar and decimal push down

1. Added enableStringsSignedMinMax parquet format plugin config and store.parquet.reader.strings_signed_min_max session option to control reading binary statistics for files generated by prior versions of Parquet 1.10.0.
2. Added ParquetReaderConfig to store configuration needed during reading parquet statistics or files.
3. Provided mechanism to enable varchar / decimal filter push down.
4. Added VersionUtil to compare Drill versions in string representation.
5. Added appropriate unit tests.

closes #1537

3 years agoDRILL-3933: Surround $QUERY variable in double-quotes to avoid asterisk expansion...
Arina Ielchiieva [Tue, 13 Nov 2018 17:53:07 +0000 (19:53 +0200)] 
DRILL-3933: Surround $QUERY variable in double-quotes to avoid asterisk expansion in sqlline script

closes #1538

3 years agoDRILL-5671: Set secure ACLs (Access Control List) for Drill ZK nodes in a secure...
karthik [Wed, 11 Jul 2018 21:01:04 +0000 (14:01 -0700)] 
DRILL-5671: Set secure ACLs (Access Control List) for Drill ZK nodes in a secure cluster

closes #1467

3 years agoDRILL-6843: Update SchemaBuilder comment to match implementation
Paul Rogers [Sun, 11 Nov 2018 00:15:33 +0000 (16:15 -0800)] 
DRILL-6843: Update SchemaBuilder comment to match implementation

The schema builder API evolved a while back. The class provides a detailed comment to explain usage. That comment was out of date. This fix updates the comment to match the code.

closes #1533

3 years agoDRILL-786: Allow CROSS JOIN syntax
Igor Guzenko [Wed, 3 Oct 2018 11:55:31 +0000 (14:55 +0300)] 
DRILL-786: Allow CROSS JOIN syntax

 1. Removed throw statement in UnsupportedOperatorsVisitor
 2. Extended UnsupportedRelOperatorException's message

closes #1488

3 years agoDRILL-6848: Duration panel in a query profile's WebUI does not open
Kunal Khatua [Tue, 13 Nov 2018 22:11:51 +0000 (14:11 -0800)] 
DRILL-6848: Duration panel in a query profile's WebUI does not open

DRILL-5571 ( PR #1531 ) accidentally introduced a bug that permanently keeps the "Duration" panel closed instead of opening it. This trivial patch fixes that.

3 years agoDrill 6735: Implement Semi-Join for the Hash-Join operator (#1522)
Boaz Ben-Zvi [Tue, 13 Nov 2018 20:58:31 +0000 (12:58 -0800)] 
Drill 6735: Implement Semi-Join for the Hash-Join operator (#1522)

3 years agoDRILL-2035: Add ability to cancel multiple queries
Kunal Khatua [Tue, 13 Nov 2018 20:12:09 +0000 (12:12 -0800)] 
DRILL-2035: Add ability to cancel multiple queries

Currently Drill UI allows canceling one query at a time.
This commit (on lines of DRILL-5571 /  PR #1531) allows for cancelling multiple `running` queries.

3 years agoDRILL-5571: Cancel running query from its Web UI
Kunal Khatua [Tue, 13 Nov 2018 20:09:42 +0000 (12:09 -0800)] 
DRILL-5571: Cancel running query from its Web UI

Currently, a running query can be cancelled by navigating to the Edit Query Tab of the profile page, making it difficult to access. This commit allows the user to cancel.
In addition, the Duration segment is also made visible by default rather than collapsed.

3 years agoDRILL-6760: Retain original exception in Verbose Error Message
shuifeng lu [Wed, 31 Oct 2018 11:54:52 +0000 (19:54 +0800)] 
DRILL-6760: Retain original exception in Verbose Error Message

closes #1519

3 years agoDRILL-6717: lower and upper functions not works with national characters
ozinoviev [Tue, 28 Aug 2018 17:29:55 +0000 (20:29 +0300)] 
DRILL-6717: lower and upper functions not works with national characters

closes #1450

3 years agoDRILL-6610: Add support for Minimum TLS restriction.
superbstreak [Wed, 18 Jul 2018 21:26:09 +0000 (14:26 -0700)] 
DRILL-6610: Add support for Minimum TLS restriction.

closes #1388

3 years agoDRILL-4456: Add Hive translate UDF
Volodymyr Vysotskyi [Thu, 1 Nov 2018 16:31:16 +0000 (18:31 +0200)] 
DRILL-4456: Add Hive translate UDF

closes #1527

3 years agoDRILL-3610: Add TIMESTAMPADD and TIMESTAMPDIFF functions
Volodymyr Vysotskyi [Wed, 7 Nov 2018 18:03:40 +0000 (20:03 +0200)] 

closes #1528

3 years agoDRILL-6837: Missing reference for dynamically created Javascript library
Kunal Khatua [Thu, 8 Nov 2018 23:13:43 +0000 (15:13 -0800)] 
DRILL-6837: Missing reference for dynamically created Javascript library

A PR for DRILL-6084 broke the auto-complete library required for Edit Query page, with the removal of the static javascript library.

The dynamically loaded script is not defined in the Freemarker template page.
The fix is trivial.

3 years agoDRILL-6611: Add Ctrl+Enter support for query submission
Kunal Khatua [Mon, 5 Nov 2018 18:23:39 +0000 (10:23 -0800)] 
DRILL-6611: Add Ctrl+Enter support for query submission

1. BugFix on parent commit: Ensure query submission is done with user name when impersonation is enabled.
2. Support non-Mac browsers
3. Support keyboard submission for profile pages with Edit Query tab.

3 years agoDRILL-6611 to enable meta-enter query submission in web query interface
boB Rudis [Mon, 23 Jul 2018 12:16:45 +0000 (08:16 -0400)] 
DRILL-6611 to enable meta-enter query submission in web query interface

3 years agoDRILL-6768: Improve to_date, to_time and to_timestamp and corresponding cast function...
Bohdan Kazydub [Wed, 3 Oct 2018 17:07:46 +0000 (20:07 +0300)] 
DRILL-6768: Improve to_date, to_time and to_timestamp and corresponding cast functions to handle empty string when  option is enabled
closes #1494

3 years agoDRILL-6809: Handle repeated map in schema inference
Paul Rogers [Sun, 28 Oct 2018 05:47:58 +0000 (22:47 -0700)] 
DRILL-6809: Handle repeated map in schema inference

It turns out that the RowSet utilities build a repeated map without including the hidden $offsets$ vector in the metadata for the map. But, other parts in Drill do include this vector.

The RowSet behavior might be a bug which can be addressed in another PR.

This PR:

* Adds unit tests for map accessors at the row set level. Looks like these were never added originally. They are a simplified form of the ResultSetLoader map tests.
* Verified that the schema inference can infer a schema from a repeated map (using the RowSet style.)
* Added a test to reproduce the case from the bug.
* Made a tweak to the RowSetBuilder to allow access to the RowSetWriter which is needed by the new tests.
* Could of minor clean-ups.

closes #1513

3 years agoDRILL-6810: Disable NULL_IF_NULL NullHandling for functions with ComplexWriter
Bohdan Kazydub [Thu, 18 Oct 2018 15:15:23 +0000 (18:15 +0300)] 
DRILL-6810: Disable NULL_IF_NULL NullHandling for functions with ComplexWriter
closes #1509

3 years agoDRILL-6824: Handle schema changes in MapRDBJsonRecordReader
Gautam Parai [Sat, 22 Sep 2018 01:44:38 +0000 (18:44 -0700)] 
DRILL-6824: Handle schema changes in MapRDBJsonRecordReader
closes #1518

3 years agoDRILL-6798: Planner changes to support semi-join.
Hanumath Rao Maduri [Wed, 10 Oct 2018 00:33:43 +0000 (17:33 -0700)] 
DRILL-6798: Planner changes to support semi-join.

3 years agoDRILL-6819: Remove invisible back link in Drill WebUI
Denys Ordynskiy [Thu, 1 Nov 2018 12:04:11 +0000 (14:04 +0200)] 
DRILL-6819: Remove invisible back link in Drill WebUI

3 years agoFixed imports for DRILL-6381
dvjyothsna [Wed, 31 Oct 2018 16:58:38 +0000 (09:58 -0700)] 
Fixed imports for DRILL-6381

3 years agoDRILL-6763: Codegen optimization of SQL functions with constant values(#1481)
shuifeng lu [Wed, 26 Sep 2018 03:22:20 +0000 (11:22 +0800)] 
DRILL-6763: Codegen optimization of SQL functions with constant values(#1481)

closes #1481

3 years agoDRILL-6811: Fix type inference to return correct data mode for boolean functions
Volodymyr Vysotskyi [Wed, 24 Oct 2018 09:57:24 +0000 (12:57 +0300)] 
DRILL-6811: Fix type inference to return correct data mode for boolean functions

closes #1510

3 years agoDRILL-6381: Address code review comments (part 3).
Aman Sinha [Sun, 14 Oct 2018 06:38:17 +0000 (23:38 -0700)] 
DRILL-6381: Address code review comments (part 3).

DRILL-6381: Add missing joinControl logic for INTERSECT_DISTINCT.

- Modified HashJoin's probe phase to process INTERSECT_DISTINCT.

- NOTE: For build phase, the functionality will be same as for SemiJoin when it is added later.

DRILL-6381: Address code review comment for intersect_distinct.

DRILL-6381: Rebase on latest master and fix compilation issues.

DRILL-6381: Generate protobuf files for C++ native client.

DRILL-6381: Use shaded Guava classes.  Add more comments and Javadoc.

3 years agoDRILL-6381: Address review comments (part 2): fix formatting issues and add javadoc.
Aman Sinha [Wed, 10 Oct 2018 23:36:48 +0000 (16:36 -0700)] 
DRILL-6381: Address review comments (part 2): fix formatting issues and add javadoc.

3 years agoDRILL-6381: Address code review comments.
Aman Sinha [Mon, 1 Oct 2018 19:06:39 +0000 (12:06 -0700)] 
DRILL-6381: Address code review comments.

3 years agoDRILL-6381: (Part 5) Update Javadoc for a few interfaces.
Aman Sinha [Thu, 13 Sep 2018 10:55:47 +0000 (03:55 -0700)] 
DRILL-6381: (Part 5) Update Javadoc for a few interfaces.

3 years agoDRILL-6381: (Part 4) Enhance MapR-DB plugin to support querying secondary indexes
rebase [Fri, 16 Mar 2018 21:24:20 +0000 (14:24 -0700)] 
DRILL-6381: (Part 4) Enhance MapR-DB plugin to support querying secondary indexes

  1. Implementation of the index descriptor for MapR-DB.
  2. MapR-DB specific costing for covering and non-covering indexes.
  3. Discovery componenent to discover the indexes available for a MapR-DB table including CAST functional indexes.
  4. Utility functions to build a canonical index descriptor.
  5. Statistics: fetch and initialize statistcs from MapR-DB for a query condition. Maintain a query-scoped cache for the statistics. Utility functions to compute selectivity.
  6. Range Partitioning: partitioning function that takes into account the tablet map to find out where a particular rowkey belongs.
  7. Restricted Scan: support doing restricted (i.e skip) scan through lookups on the rowkey. Added a group-scan and record reader for this.
  8. MD-3726: Simple Order by queries (without limit) when an index is used are showing regression.
  9. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON
  10. MD-4259 : Account for limit during hashcode computation

Co-authored-by: Aman Sinha <asinha@maprtech.com>
Co-authored-by: chunhui-shi <cshi@maprtech.com>
Co-authored-by: Gautam Parai <gparai@maprtech.com>
Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>
Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Fix additional compilation issues.

3 years agoDRILL-6381: (Part 3) Planner and Execution implementation to support Secondary Indexes
rebase [Fri, 16 Mar 2018 21:25:26 +0000 (14:25 -0700)] 
DRILL-6381: (Part 3) Planner and Execution implementation to support Secondary Indexes

  1. Index Planning Rules and Plan generators
    - DbScanToIndexScanRule: Top level physical planning rule that drives index planning for several relational algebra patterns.
    - DbScanSortRemovalRule: Physical planning rule for index planning for Sort-based operations.
    - Plan Generators: Covering, Non-Covering and Intersect physical plan generators.
    - Support planning with functional indexes such as CAST functions.
    - Enhance PlannerSettings with several configuration options for indexes.
  2. Index Selection and Statistics
    - An IndexSelector that support cost-based index selection of covering and non-covering indexes using statistics and collation properties.
    - Costing of index intersection for comparison with single-index plans.
  3. Planning and execution operators
    - Support RangePartitioning physical operator during query planning and execution.
    - Support RowKeyJoin physical operator during query planning and execution.
    - HashTable and HashJoin changes to support RowKeyJoin and Index Intersection.
    - Enhance Materializer to keep track of subscan association with a particular rowkey join.
  4. Index Planning utilities
    - Utility classes to perform RexNode analysis, including conversion to and from SchemaPath.
    - Utility class to analyze filter condition and an input collation to determine output collation.
    - Helper classes to maintain index contexts for logical and physical planning phase.
    - IndexPlanUtils utility class for various helper methods.
  5. Miscellaneous
    - Separate physical rel for DirectScan.
    - Modify LimitExchangeTranspose rule to handle SingleMergeExchange.
    - MD-3880: Return correct status from RangePartitionRecordBatch setupNewSchema

Co-authored-by: Aman Sinha <asinha@maprtech.com>
Co-authored-by: chunhui-shi <cshi@maprtech.com>
Co-authored-by: Gautam Parai <gparai@maprtech.com>
Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>
Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

Resolve merge comflicts and compilation issues.

3 years agoDRILL-6381: (Part 2) MapRDB plugin update to 6.0.1
rebase [Mon, 12 Feb 2018 22:31:49 +0000 (14:31 -0800)] 
DRILL-6381: (Part 2) MapRDB plugin update to 6.0.1

  1. MD-3960: Update Drill to build with MapR-6.0.1 libraries
  2. MD-3995: Do not pushdown limit 0 past project with CONVERT_FROMJSON
  3. MD-4054: Restricted scan limit is changed to dynamically read rows using the rowcount of the rightside instead of 4096.
  4. MD-3688: Impersonating a view owner doesn't work with security disabled in 6.0
  5. MD-4492: Missing limit pushdown changes in JsonTableGroupScan

Co-authored-by: chunhui-shi <cshi@maprtech.com>
Co-authored-by: Gautam Parai <gparai@maprtech.com>
Co-authored-by: Vlad Rozov <vrozov@mapr.com>

3 years agoDRILL-6381: (Part 1) Secondary Index framework
rebase [Mon, 12 Feb 2018 22:10:56 +0000 (14:10 -0800)] 
DRILL-6381: (Part 1) Secondary Index framework

  1. Secondary Index planning interfaces and abstract classes like DBGroupScan, DbSubScan, IndexDecriptor etc.
  2. Statistics and Cost model interfaces/classes: PluginCost, Statistics, StatisticsPayload, AbstractIndexStatistics
  3. ScanBatch and RecordReader to support repeatable scan
  4. Secondary Index execution related interfaces: RangePartitionSender, RowKeyJoin, PartitionFunction
  5. MD-3979: Query using cast index plan fails with NPE

Co-authored-by: Aman Sinha <asinha@maprtech.com>
Co-authored-by: chunhui-shi <cshi@maprtech.com>
Co-authored-by: Gautam Parai <gparai@maprtech.com>
Co-authored-by: Padma Penumarthy <ppenumar97@yahoo.com>
Co-authored-by: Hanumath Rao Maduri <hmaduri@maprtech.com>

3 years agoDRILL-6715: Update descriptions for System Options table
Kunal Khatua [Thu, 25 Oct 2018 19:08:57 +0000 (12:08 -0700)] 
DRILL-6715: Update descriptions for System Options table

With introduction of DRILL-5735 , the descriptions for about half the system options still remain missing. This commit collects descriptions review by @bbevens

1. Update options for HashAgg/Join (@Ben-Zvi )
2. Update options for Parquet Reader/Writer (@sachouche )
3. Update options for Planners (@HanumathRao , @vdiravka , @KazydubB )
4. Update options for BatchSizing (@bitblender )
5. Update options for Planner Optimizations (@arina-ielchiieva )
6. Update options for Security & Kafka (Krystal Nguyen)
7. Update options for Misc entries (@arina-ielchiieva , @vvysotskyi )

In additional, there is a patch for `org.apache.drill.exec.compile.ClassTransformer.scalar_replacement` , which appears to have replaced `exec.compile.scalar_replacement`. References to the latter have been removed to avoid confusion.
Additional changes include moving the `ClassTransformer` validator to `ExecConstants.java`
Adding support for internal options' descriptions
Removed mention of {{Will be removed in 1.15.0}}. (Refer DRILL-6527)

3 years agoDRILL-6804: Simplify usage of OperatorPhase in HashAgg.
Timothy Farkas [Thu, 18 Oct 2018 23:32:03 +0000 (16:32 -0700)] 
DRILL-6804: Simplify usage of OperatorPhase in HashAgg.

3 years agoDRILL-6719: Separate spilling queue logic from HashJoin and HashAgg.
Timothy Farkas [Tue, 28 Aug 2018 23:28:49 +0000 (16:28 -0700)] 
DRILL-6719: Separate spilling queue logic from HashJoin and HashAgg.

3 years agoDRILL-6795: Upgrade Janino compiler from 2.7.6 to 3.0.10
Volodymyr Vysotskyi [Sat, 13 Oct 2018 13:13:36 +0000 (16:13 +0300)] 
DRILL-6795: Upgrade Janino compiler from 2.7.6 to 3.0.10

closes #1503

3 years agoDRILL-6793: FragmentExecutor cannot send its final state for the case when RootExec...
Bohdan Kazydub [Tue, 16 Oct 2018 14:11:36 +0000 (17:11 +0300)] 
DRILL-6793: FragmentExecutor cannot send its final state for the case when RootExec root wasn't initialized

closes #1506

3 years agoDRILL-6785: DataClient is using RootAllocator in the bootstrap instead of dataPool
Sorabh Hamirwasia [Mon, 8 Oct 2018 18:28:27 +0000 (11:28 -0700)] 
DRILL-6785: DataClient is using RootAllocator in the bootstrap instead of dataPool

closes #1502

3 years agoDRILL-6084: Show Drill functions in WebUI for autocomplete
Kunal Khatua [Thu, 18 Oct 2018 00:30:03 +0000 (17:30 -0700)] 
DRILL-6084: Show Drill functions in WebUI for autocomplete

Building on top of DRILL-3988 and leveraging DRILL-5868, this allows support for Drill functions to be now available in the WebUI.
If users wants UDFs to show up, they should place the UDF jars in the `$DRILL_HOME/jars/3rdparty` directory so that this can be loaded during the Drillbit's startup.
Concept of internal Drill functions are introduced. With this, internal Drill functions like `ConvertToNullableXYZ` has been marked as internal.
The WebUI will not show these functions. However, they are still visible in `sys.functions` table with an additional column indicating that it  is an internal function.
Tests have been added as a part of this commit to verify the internal functions concept.

3 years agoDRILL-6797: Fix UntypedNull handling for complex types
Arina Ielchiieva [Thu, 11 Oct 2018 16:25:17 +0000 (19:25 +0300)] 
DRILL-6797: Fix UntypedNull handling for complex types

3 years agoDRILL-6775: The schema for empty output is not shown in Drill Web UI
Anton Gozhiy [Tue, 9 Oct 2018 12:59:16 +0000 (15:59 +0300)] 
DRILL-6775: The schema for empty output is not shown in Drill Web UI

Removed an excess check for the result emptiness that prevented retrieval of the column names.

close apache/drill#1498

3 years agoDRILL-6783: CAST string literal as INTERVAL MONTH/YEAR works inconsistently when...
Bohdan Kazydub [Mon, 8 Oct 2018 13:50:22 +0000 (16:50 +0300)] 
DRILL-6783: CAST string literal as INTERVAL MONTH/YEAR works inconsistently when selecting from a table with multiple rows

close apache/drill#1496

3 years agoDRILL-6777: Setup CircleCI configs for Drill
Vitalii Diravka [Tue, 11 Sep 2018 11:39:13 +0000 (14:39 +0300)] 
DRILL-6777: Setup CircleCI configs for Drill

- adding .circleci/config.yml to the project to launch CircleCI
- custom memory parameters
- usage of CircleCI machine
- excluding "SlowTest" and "UnlikelyTest" groups
- update maven version
- adding libaio.so library to solve MySQL integration tests
- update com.jcabi:jcabi-mysql-maven-plugin library version
- TODO descriptions for the future enhancements of CircleCI build for Drill

close apache/drill#1493

3 years agoDRILL-6473: Update MapR Hive
Bohdan Kazydub [Mon, 21 May 2018 16:11:23 +0000 (19:11 +0300)] 
DRILL-6473: Update MapR Hive

close apache/drill#1307

3 years agoDRILL-3988: Expose Drill built-in functions & UDFs in a system table (#1483)
Kunal Khatua [Thu, 11 Oct 2018 17:30:15 +0000 (10:30 -0700)] 
DRILL-3988: Expose Drill built-in functions & UDFs  in a system table (#1483)

This commit exposes available SQL functions in Drill and also detects UDFs that have been dynamically loaded into Drill.
An example is shown below for 2 UDFs dynamically loaded into the cluster, along side the existing built-in functions that come with Drill.

0: jdbc:drill:schema=sys> select source, count(*) as functionCount from sys.functions group by source;
|                 source                  | functionCount  |
| built-in                                | 2704           |
| simple-drill-function-1.0-SNAPSHOT.jar  | 12             |
| drill-url-tools-1.0.jar                 | 1              |
3 rows selected (0.209 seconds)

The system table exposes information as shown. The UDF is initialized, making the `returnType` available.
The `random(FLOAT8-REQUIRED,FLOAT8-REQUIRED)` function is an example of a UDF that has overloaded arguments (see `signature`).
The `url_parse(VARCHAR-REQUIRED)` function is another example of an initialized UDF.
Rest are built-in functions that meet the query's filter criteria.
0: jdbc:drill:schema=sys> select * from sys.functions where name like 'random' or name like '%url%';
|    name     |            signature             | returnType  |                 source                  |
| parse_url   | VARCHAR-REQUIRED                 | LATE        | built-in                                |
| random      |                                  | FLOAT8      | built-in                                |
| random      | FLOAT8-REQUIRED,FLOAT8-REQUIRED  | FLOAT8      | simple-drill-function-1.0-SNAPSHOT.jar  |
| url_decode  | VARCHAR-REQUIRED                 | VARCHAR     | built-in                                |
| url_encode  | VARCHAR-REQUIRED                 | VARCHAR     | built-in                                |
| url_parse   | VARCHAR-REQUIRED                 | LATE        | drill-url-tools-1.0.jar                 |
6 rows selected (0.619 seconds)

3 years agoDRILL-6776: Removed external jQuery dependency
Igor Guzenko [Mon, 8 Oct 2018 10:55:03 +0000 (13:55 +0300)] 
DRILL-6776: Removed external jQuery dependency

3 years agoDRILL-6731: use thread pool to run the runtime filter aggregating work
weijie.tong [Tue, 9 Oct 2018 12:28:22 +0000 (20:28 +0800)] 
DRILL-6731: use thread pool to run the runtime filter aggregating work
closes #1459

3 years agoDRILL-6731: Resolving race conditions in RuntimeFilterSink
Sorabh Hamirwasia [Wed, 26 Sep 2018 20:20:02 +0000 (13:20 -0700)] 
DRILL-6731: Resolving race conditions in RuntimeFilterSink
Add condition variable to avoid starvation of producer thread while acquiring queue lock