drill.git
5 years ago[maven-release-plugin] prepare release drill-1.9.0 1.9.0 drill-1.9.0
Sudheesh Katkam [Fri, 18 Nov 2016 19:44:30 +0000 (11:44 -0800)] 
[maven-release-plugin] prepare release drill-1.9.0

5 years agoRevert "DRILL-4373: Drill and Hive have incompatible timestamp representations in...
Sudheesh Katkam [Fri, 11 Nov 2016 23:37:35 +0000 (15:37 -0800)] 
Revert "DRILL-4373: Drill and Hive have incompatible timestamp representations in parquet - added sys/sess option "store.parquet.int96_as_timestamp"; - added int96 to timestamp converter for both readers; - added unit tests;"

This reverts commit 7e7214b40784668d1599f265067f789aedb6cf86.

5 years agoDRILL-5047: When session option is string, query profile is displayed incorrectly...
Arina Ielchiieva [Thu, 17 Nov 2016 12:44:34 +0000 (12:44 +0000)] 
DRILL-5047: When session option is string, query profile is displayed incorrectly on Web UI

closes #655

5 years agoDRILL-5009: Skip reading of empty row groups while reading Parquet metadata 658/head
Parth Chandra [Tue, 8 Nov 2016 04:29:23 +0000 (20:29 -0800)] 
DRILL-5009: Skip reading of empty row groups while reading Parquet metadata

+ We will no longer attempt to scan such row groups.

closes #651

5 years agoDRILL-5007: Dynamic UDF lazy-init does not work correctly in multi-node cluster
Arina Ielchiieva [Wed, 9 Nov 2016 11:08:18 +0000 (13:08 +0200)] 
DRILL-5007: Dynamic UDF lazy-init does not work correctly in multi-node cluster

closes #650

5 years agoAdd Sudheesh's PGP Key
Sudheesh Katkam [Sat, 5 Nov 2016 19:37:10 +0000 (12:37 -0700)] 
Add Sudheesh's PGP Key

5 years agoDRILL-4823: Fix OOM while trying to prune partitions with reasonable data size
Arina Ielchiieva [Wed, 3 Aug 2016 11:56:43 +0000 (11:56 +0000)] 
DRILL-4823: Fix OOM while trying to prune partitions with reasonable data size

closes #560

5 years agoDRILL-4674: Allow casting to boolean the same literals as in Postgres
Arina Ielchiieva [Tue, 11 Oct 2016 16:30:14 +0000 (19:30 +0300)] 
DRILL-4674: Allow casting to boolean the same literals as in Postgres

closes #610

5 years agoDRILL-4969: Basic implementation of display size for column metadata
Laurent Goujon [Thu, 27 Oct 2016 00:17:08 +0000 (17:17 -0700)] 
DRILL-4969: Basic implementation of display size for column metadata

Add a simple implementation of display size, based on the ODBC table
available at: https://msdn.microsoft.com/en-us/library/ms713974(v=vs.85).aspx

closes #632

5 years agoDRILL-4945: Report INTERVAL exact type as column type name
Laurent Goujon [Fri, 14 Oct 2016 18:27:28 +0000 (11:27 -0700)] 
DRILL-4945: Report INTERVAL exact type as column type name

closes #618

5 years agoDRILL-1950: Parquet rowgroup level filter pushdown in query planning time.
Jinfeng Ni [Fri, 4 Nov 2016 21:56:52 +0000 (14:56 -0700)] 
DRILL-1950: Parquet rowgroup level filter pushdown in query planning time.

Implement Parquet rowgroup level filter pushdown. The filter pushdown is performed in
in Drill physical planning phase.

Only a local filter, which refers to columns in a single table, is qualified for filter pushdown.

A filter may be qualified if it is a simple comparison filter, or a compound "and/or" filter consists of
simple comparison filter. Data types allowed in comparison filter are int, bigint, float, double, date,
timestamp, time. Comparison operators are =, !=, <, <=, >, >=. Operands have to be a column of the above
data types, or an explicit cast or implicit cast function, or a constant expressions.

This closes #637

5 years agoDRILL-1950: Update parquet metadata cache format to include both min/max and addition...
Jinfeng Ni [Mon, 31 Oct 2016 04:30:12 +0000 (21:30 -0700)] 
DRILL-1950: Update parquet metadata cache format to include both min/max and additional column type information.

    Parquet meta cache format change:
    1. include both min/max in ColumnMetaData if column statistics is available,
    2. include precision/scale/repetitionLevel/definitionLevel in ColumnTypeMetaData (precision/scale/definitionLevel is for future use).

5 years agoDRILL-1950: Initial prototype patch for parquet filter pushdown.
AdamPD [Thu, 30 Jun 2016 21:09:53 +0000 (14:09 -0700)] 
DRILL-1950: Initial prototype patch for parquet filter pushdown.

Use three new classes from Adam's patch.

5 years agoDRILL-5400: Fix NPE in corrupt date detection
Parth Chandra [Fri, 4 Nov 2016 18:05:40 +0000 (11:05 -0700)] 
DRILL-5400: Fix NPE in corrupt date detection

This closes #646

5 years agoDRILL-4800: Various fixes. Fix buffer underflow exception in BufferedDirectBufInputSt...
Parth Chandra [Wed, 14 Sep 2016 04:47:49 +0000 (21:47 -0700)] 
DRILL-4800: Various fixes. Fix buffer underflow exception in BufferedDirectBufInputStream. Fix writer index for in64 dictionary encoded types. Added logging to help debug. Fix memory leaks. Work around issues with of InputStream.available() ( Do not use hasRemainder; Remove check for EOF in BufferedDirectBufInputStream.read() ). Finalize defaults. Remove commented code.
Addressed review comments

This closes #611

5 years agoDRILL-4800: Parallelize column reading. Read/Decode fixed width fields in parallel...
Parth Chandra [Wed, 24 Aug 2016 17:46:37 +0000 (10:46 -0700)] 
DRILL-4800: Parallelize column reading. Read/Decode fixed width fields in parallel Decoding var length columns in parallel Use simplified decompress method for Gzip and Snappy decompression. Avoids concurrency issue with Parquet decompression. (It's also faster). Stress test Parquet read write Parallel column reader is disabled by default (may perform less well under higher concurrency)

5 years agoDRILL-4800: Add AsyncPageReader to pipeline PageRead Use non tracking input stream...
Parth Chandra [Tue, 27 Sep 2016 21:03:35 +0000 (14:03 -0700)] 
DRILL-4800: Add AsyncPageReader to pipeline PageRead Use non tracking input stream for Parquet scans. Make choice between async and sync reader configurable. Make various options user configurable - choose between sync and async page reader, enable/disable fadvise Add Parquet Scan metrics to track time spent in various operations

5 years agoDRILL-4800: Use a buffering input stream in the Parquet reader
Parth Chandra [Fri, 10 Jun 2016 21:56:41 +0000 (14:56 -0700)] 
DRILL-4800: Use a buffering input stream in the Parquet reader

5 years agoDRILL-4989: Fix TestParquetWriter.testImpalaParquetBinaryAsTimeStamp_DictChange
Vitalii Diravka [Wed, 2 Nov 2016 15:51:56 +0000 (15:51 +0000)] 
DRILL-4989: Fix TestParquetWriter.testImpalaParquetBinaryAsTimeStamp_DictChange

closes #643

5 years agoDRILL-4986: Allow users to customize the Drill log file name
Paul Rogers [Tue, 1 Nov 2016 18:26:03 +0000 (11:26 -0700)] 
DRILL-4986: Allow users to customize the Drill log file name

Adds a new user-settable environment variable to choose the log file
base name.

closes #641

5 years agoDRILL-4951: Do Guava.patch earlier so we can run single Hbase's unit test through...
chunhui-shi [Tue, 18 Oct 2016 20:28:00 +0000 (13:28 -0700)] 
DRILL-4951: Do Guava.patch earlier so we can run single Hbase's unit test through command line or IDE

closes #636

5 years agoDRILL-4792: Include session options used for a query as part of the profile
Arina Ielchiieva [Tue, 19 Jul 2016 16:54:18 +0000 (16:54 +0000)] 
DRILL-4792: Include session options used for a query as part of the profile

closes #551

5 years agoDRILL-4927 (part 2): Add support for Null Equality Joins (mixed comparators)
Roman Kulyk [Fri, 28 Oct 2016 13:26:53 +0000 (13:26 +0000)] 
DRILL-4927 (part 2): Add support for Null Equality Joins (mixed comparators)

This changes are a subset of the original pull request from DRILL-4539 (PR-462).
- Added changes to support mixed comparators;
- Added tests for it.

closes #635

5 years agoDRILL-1996: Add cancel method to Drill C++ connector
Laurent Goujon [Tue, 11 Oct 2016 23:35:18 +0000 (16:35 -0700)] 
DRILL-1996: Add cancel method to Drill C++ connector

This closes #602

5 years agoDRILL-4420: C++ API for metadata access and prepared statements
Laurent Goujon [Mon, 15 Aug 2016 21:17:35 +0000 (14:17 -0700)] 
DRILL-4420: C++ API for metadata access and prepared statements

Add support to the C++ client for metadata querying and prepared
statement requests.

Part of the metadata API, add methods to query for server capabilities.
As of now, this interface is not backed up by any RPC exchange so
the information is pretty much static, and match Drill 1.8.0
current capabilities.

5 years agoDRILL-1268: Add unit test to C++ native client
Laurent Goujon [Mon, 22 Aug 2016 21:54:19 +0000 (14:54 -0700)] 
DRILL-1268: Add unit test to C++ native client

Add CppUnit unit test to the C++ native client

5 years agoDRILL-4853: Update C++ protobuf source files
Laurent Goujon [Thu, 18 Aug 2016 16:55:59 +0000 (09:55 -0700)] 
DRILL-4853: Update C++ protobuf source files

Add support for prepared statements and metadata querying

5 years agoDRILL-4373: Drill and Hive have incompatible timestamp representations in parquet...
Vitalii Diravka [Fri, 2 Sep 2016 21:43:50 +0000 (21:43 +0000)] 
DRILL-4373: Drill and Hive have incompatible timestamp representations in parquet - added sys/sess option "store.parquet.int96_as_timestamp"; - added int96 to timestamp converter for both readers; - added unit tests;

This closes #600

5 years agoDRILL-3243: Added CSG mods. Fixed field names.
cgivre [Thu, 6 Oct 2016 03:29:26 +0000 (23:29 -0400)] 
DRILL-3243: Added CSG mods. Fixed field names.
Removed old test files
Added Parse_url and parse_query() functions
Fix unit test

This closes #607

5 years agoDRILL-3423: Adding HTTPd Log Parsing functionality including full pushdown, type...
Jim Scott [Tue, 3 Nov 2015 21:43:54 +0000 (15:43 -0600)] 
DRILL-3423: Adding HTTPd Log Parsing functionality including full pushdown, type remapping and wildcard support.
 Pushed through the requested columns for push down to the parser. Added more tests to cover a few more use cases. Ensured that user query fields are now completely consistent with returned values.

5 years agoDRILL-3423: Initial HTTPD log plugin. Needs tests. Would be good to improve the times...
Jacques Nadeau [Tue, 28 Jul 2015 22:29:32 +0000 (15:29 -0700)] 
DRILL-3423: Initial HTTPD log plugin. Needs tests. Would be good to improve the timestamp and cookies behaviors since we can make those more type specific.

5 years agoDRILL-4884: Fix IOB exception in limit n query when n is beyond 65535.
hongze.zhz [Fri, 9 Sep 2016 08:19:16 +0000 (16:19 +0800)] 
DRILL-4884: Fix IOB exception in limit n query when n is beyond 65535.

close apache/drill#584

5 years agoDRILL-4967: Adding template_name to source code generated using freemarker template.
Jinfeng Ni [Thu, 8 Sep 2016 22:50:43 +0000 (15:50 -0700)] 
DRILL-4967: Adding template_name to source code generated using freemarker template.

close apache/drill#629

5 years agoDRILL-4974: Add missing null check in FindPartitionConditions.analyzeCall()
karthik [Thu, 20 Oct 2016 20:43:17 +0000 (13:43 -0700)] 
DRILL-4974: Add missing null check in FindPartitionConditions.analyzeCall()

close apache/drill#634

5 years agoDRILL-4972: Remove setDaemon(true) call in WorkManager.StatusThread
Sudheesh Katkam [Fri, 28 Oct 2016 16:08:23 +0000 (09:08 -0700)] 
DRILL-4972: Remove setDaemon(true) call in WorkManager.StatusThread

closes #633

5 years agoDRILL-4964: Make Drill reconnect to hive metastore after hive metastore is restarted.
Sorabh Hamirwasia [Wed, 26 Oct 2016 01:52:01 +0000 (18:52 -0700)] 
DRILL-4964: Make Drill reconnect to hive metastore after hive metastore is restarted.

Drill fails to connect to hive metastore after hive metastore is restarted unless drillbits are restarted.

Changes: For methods DrillHiveMetaStoreClient.getAllDatabases() and DrillHiveMetaStoreClient.getAllTables(),
the HiveMetaStoreClient wraps MetaException and TException both into MetaException. In case of connection
failure which is thrown as TException it is difficult to categorize at DrillClient level. The fix is to
close older connection and reconnect in case of these 2 api's. In all other cases proper set of exceptions
are thrown where we can handle each one individually.

close apache/drill#628

5 years agoDRILL-4826: Query against INFORMATION_SCHEMA.TABLES degrades as the number of views
Parth Chandra [Thu, 4 Aug 2016 06:02:01 +0000 (23:02 -0700)] 
DRILL-4826: Query against INFORMATION_SCHEMA.TABLES degrades as the number of views
increases

This closes #592

5 years agoDRILL-4968: Add column size to ColumnMetadata
Laurent Goujon [Wed, 26 Oct 2016 22:17:33 +0000 (15:17 -0700)] 
DRILL-4968: Add column size to ColumnMetadata

Add a column size to ColumnMetadata so that JDBC and ODBC clients share
the implementation and don't have to recompute it client side.

this closes #631

5 years agoDRILL-4560: When new bits register, invoke listeners in ZKClusterCoordinator
Paul Rogers [Wed, 19 Oct 2016 19:15:12 +0000 (12:15 -0700)] 
DRILL-4560: When new bits register, invoke listeners in ZKClusterCoordinator

close apache/drill#626

Original JIRA title: “ZKClusterCoordinator does not call
DrillbitStatusListener.drillbitRegistered for new bits”

5 years agoDRILL-4905: Push down the LIMIT to the parquet reader scan to limit the numbers of...
Padma Penumarthy [Mon, 17 Oct 2016 23:46:51 +0000 (16:46 -0700)] 
DRILL-4905: Push down the LIMIT to the parquet reader scan to limit the numbers of records read

close apache/drill#597

5 years agoDRILL-4954: Fix printing Strings when `allTextMode` is ON for MapR-DB JSON tables.
Smidth Panchamia [Thu, 20 Oct 2016 00:29:31 +0000 (17:29 -0700)] 
DRILL-4954: Fix printing Strings when `allTextMode` is ON for MapR-DB JSON tables.

close apache/drill#624

5 years agoDRILL-4921: Added -P to "cd" in scripts to allow for symbolic links
Boaz Ben-Zvi [Tue, 18 Oct 2016 23:01:11 +0000 (16:01 -0700)] 
DRILL-4921: Added -P to "cd" in scripts to allow for symbolic links

close apache/drill#623

5 years agoDRILL-4950: Remove incorrect false condition; consume all empty batches
Sudheesh Katkam [Tue, 18 Oct 2016 21:22:53 +0000 (14:22 -0700)] 
DRILL-4950: Remove incorrect false condition; consume all empty batches

closes #621

5 years agoDRILL-4369: Exchange name and version infos during handshake
Laurent Goujon [Tue, 18 Oct 2016 22:01:38 +0000 (15:01 -0700)] 
DRILL-4369: Exchange name and version infos during handshake

There's no name and version exchanged between client and server over the User RPC
channel.

On client side, having access to the server name and version is useful to expose it
to the user (through JDBC or ODBC api like DatabaseMetadata#getDatabaseProductVersion()),
or to implement fallback strategy when some recent API are not available (like
metadata API).

On the server side, having access to the client version might be useful for audit
purposes and eventually to implement fallback strategy if it doesn't require a RPC
version change.

this closes #622

5 years agoDRILL-4653: Malformed JSON should not stop the entire query from progressing
Subbu Srinivasan [Fri, 10 Jun 2016 22:58:49 +0000 (15:58 -0700)] 
DRILL-4653: Malformed JSON should not stop the entire query from progressing

This closes #518

5 years agoDRILL-3178: csv reader should allow newlines inside quotes
Fran\9dcois Methot [Wed, 21 Sep 2016 00:16:15 +0000 (20:16 -0400)] 
DRILL-3178: csv reader should allow newlines inside quotes

This closes #593

5 years agoDRILL-4726: Dynamic UDF Support
Arina Ielchiieva [Mon, 22 Aug 2016 13:29:30 +0000 (13:29 +0000)] 
DRILL-4726: Dynamic UDF Support

1) Configuration / parsing / options / protos
2) Zookeeper integration
3) Registration / unregistration / lazy-init
4) Unit tests

This closes #574

5 years agoDRILL-4203: Parquet File. Date is stored wrongly - Added new extra field in the parqu...
Vitalii Diravka [Mon, 19 Sep 2016 12:59:27 +0000 (12:59 +0000)] 
DRILL-4203: Parquet File. Date is stored wrongly - Added new extra field in the parquet meta info "is.date.correct = true"; - Removed unnecessary double conversion of value with Julian day; - Added ability to correct corrupted dates for parquet files with the second version old metadata cache file as well.

This closes #595

5 years agoDRILL-4203: Fix date values written in parquet files created by Drill
Jason Altekruse [Thu, 31 Dec 2015 16:22:04 +0000 (10:22 -0600)] 
DRILL-4203: Fix date values written in parquet files created by Drill

Drill was writing non-standard dates into parquet files for all releases
before 1.9.0. The values have been read by Drill correctly by Drill, but
external tools like Spark reading the files will see corrupted values for
all dates that have been written by Drill.

This change corrects the behavior of the Drill parquet writer to correctly
store dates in the format given in the parquet specification.

To maintain compatibility with old files, the parquet reader code has
been updated to check for the old format and automatically shift the
corrupted values into corrected ones automatically.

The test cases included here should ensure that all files produced by
historical versions of Drill will continue to return the same values they
had in previous releases. For compatibility with external tools, any old
files with corrupted dates can be re-written using the CREATE TABLE AS
command (as the writer will now only produce the specification-compliant
values, even if after reading out of older corrupt files).

While the old behavior was a consistent shift into an unlikely range
to be used in a modern database (over 10,000 years in the future), these are still
valid date values. In the case where these may have been written into
files intentionally, and we cannot be certain from the metadata if Drill
produced the files, an option is included to turn off the auto-correction.
Use of this option is assumed to be extremely unlikely, but it is included
for completeness.

This patch was originally written against version 1.5.0, when rebasing
the corruption threshold was updated to 1.9.0.

Added regenerated binary files, updated metadata cache files accordingly.

One small fix in the ParquetGroupScan to accommodate changes in master that changed
when metadata is read.

Tests for bugs revealed by the regression suite.

Fix drill version number in metadata file generation

5 years agoDRILL-4203: Fix DrillVersionInfo to make it provide a valid version number even durin...
Jason Altekruse [Wed, 27 Jan 2016 18:20:01 +0000 (10:20 -0800)] 
DRILL-4203: Fix DrillVersionInfo to make it provide a valid version number even during the unit tests.

This is now a build-time generated class, rather than one that looks on the
classpath for META-INF files.

This pattern for file generation with parameters passed from the POM files
was borrowed from parquet-mr.

5 years agoDRILL-4870 drill-config.sh sets JAVA_HOME incorrectly for the Mac
Paul Rogers [Fri, 7 Oct 2016 20:53:38 +0000 (13:53 -0700)] 
DRILL-4870 drill-config.sh sets JAVA_HOME incorrectly for the Mac

This closes #605

5 years agoDRILL-4925: Add tableType filter to GetTables metadata query
Laurent Goujon [Tue, 27 Sep 2016 20:37:31 +0000 (13:37 -0700)] 
DRILL-4925: Add tableType filter to GetTables metadata query

- Adding tableType filter to GetTablesReq query (needed for JDBC and ODBC
drivers).
- Fix table type returned by sys and INFORMATION_SCHEMA tables
- Also fixes some protobuf typos to related classes.

this closes #612

Change-Id: If95246a312f6c6d64a88872936f516308874c2d2

5 years agoDRILL-4930: Fix Metadata results ordering
Laurent Goujon [Tue, 4 Oct 2016 23:35:50 +0000 (16:35 -0700)] 
DRILL-4930: Fix Metadata results ordering

Change MetadataProvider to return metadata results ordered (following
convention used by ODBC and JDBC specs).

this closes #614

Change-Id: Iff59b7fada7040602f1735bccc13bc6bf5c9a252

5 years agoDRILL-4880: Support JDBC driver registration using ServiceLoader
Laurent Goujon [Tue, 27 Sep 2016 00:14:59 +0000 (17:14 -0700)] 
DRILL-4880: Support JDBC driver registration using ServiceLoader

Support loading Drill driver using ServiceLoader. From the user perspective,
it means being able to use the driver without registering it first, like by using
Class.forName("org.apache.drill.jdbc.Driver") for example.

this closes #596

Change-Id: Id26922ee42bef5fbce46ac2bcbb83f1859e9bb48

5 years agoDRILL-4452: Uses Apache Calcite Avatica driver vs Optiq driver for Drill JDBC
Laurent Goujon [Mon, 15 Aug 2016 23:57:23 +0000 (16:57 -0700)] 
DRILL-4452: Uses Apache Calcite Avatica driver vs Optiq driver for Drill JDBC

Drill JDBC driver uses Optiq Avatica as its basis, but this dependency has
been moved over to Calcite, for quite some time without Drill code being
updated for it.

This patch updates Avatica version to the version from Calcite
(1.4.0-drill-r19). It also refactors Drill JDBC driver to comply with the
packages and API changes in Avatica. Finally it fixes the the SQL types for
lists and structs, since Drill doesn't support java.sql.Array and
java.sql.Struct interfaces.

this closes #395

Change-Id: Ia608adf900e8708d9e6f6f58ed41e104321a9914

5 years agoDRILL-4927: Add support for Null Equality Joins
Roman Kulyk [Tue, 4 Oct 2016 13:58:13 +0000 (13:58 +0000)] 
DRILL-4927: Add support for Null Equality Joins

These changes are a subset of the original pull request from DRILL-4539 (PR-462).
- Added changes to support Null Equality Joins;
- Created tests for it.

close apache/drill#603

5 years agoDRILL-4862: Fix binary_string to use an injected buffer as out buffer
chunhui-shi [Thu, 25 Aug 2016 17:23:53 +0000 (10:23 -0700)] 
DRILL-4862: Fix binary_string to use an injected buffer as out buffer

+ Previously, binary_string used the input buffer as output buffer. So after calling binary_string, the original content was destroyed. Other expressions/ functions that need to access the original input buffer get wrong results.
+ This fix also sets readerIndex and writerIndex correctly for the output buffer, otherwise the consumer of the output buffer will hit issues.

closes #604

5 years agoDRILL-4618: Correct the usage of random flag in Hive function registry
chunhui-shi [Thu, 26 May 2016 06:22:01 +0000 (23:22 -0700)] 
DRILL-4618: Correct the usage of random flag in Hive function registry

+ Function visitor should not use previous function holder if this function is non-deterministic

closes #509

5 years agoDRILL-4911: Avoid plan serialization in SimpleParallelizer when debug logging is... 599/head
Jinfeng Ni [Wed, 21 Sep 2016 23:03:49 +0000 (16:03 -0700)] 
DRILL-4911:  Avoid plan serialization in SimpleParallelizer when debug logging is not enabled.

5 years agoDRILL-4906: CASE Expression with constant generates class exception
Serhii-Harnyk [Thu, 22 Sep 2016 12:06:10 +0000 (15:06 +0300)] 
DRILL-4906: CASE Expression with constant generates class exception

Tests for different data types

close apache/drill#598

5 years agoDRILL-4771: Drill should avoid doing the same join twice if count(distinct) exists
Gautam Parai [Tue, 13 Sep 2016 03:21:46 +0000 (20:21 -0700)] 
DRILL-4771: Drill should avoid doing the same join twice if count(distinct) exists

close apache/drill#588

5 years agoDRILL-3898 : Sort spill was modified to catch all errors, ignore repeated errors...
Boaz Ben-Zvi [Fri, 9 Sep 2016 23:36:03 +0000 (16:36 -0700)] 
DRILL-3898 :  Sort spill was modified to catch all errors, ignore repeated errors while closing the new group and issue a more detailed error message.

close apache/drill#591

5 years agoDRILL-4894: Fix unit test failure in 'storage-hive/core' module 587/head
Aditya Kishore [Fri, 16 Sep 2016 19:14:35 +0000 (12:14 -0700)] 
DRILL-4894: Fix unit test failure in 'storage-hive/core' module

Exclude 'hadoop-mapreduce-client-core' and 'hadoop-auth' as transitive dependencies from 'hbase-server'

5 years agoDRILL-4886: Modifying projects POMs to align with Drill's build and distribution. 586/head
Aditya Kishore [Sun, 11 Sep 2016 20:23:23 +0000 (13:23 -0700)] 
DRILL-4886: Modifying projects POMs to align with Drill's build and distribution.

The default build/test/packaging behavior for mapr-format-plugin module are

1. BUILD of mapr-format-plugin is ENABLED.
2. Unit tests of mapr-format-plugin module are DISABLED (use `-Pmapr` to enable).
3. Packaging of mapr-format-plugin is DISABLED (use `-Pmapr` to enable).

Please see LEGAL-251 for discussion/conclusion regarding inclusion of source code with non-open-source dependency.

5 years agoExclude 'drill-memory-base' and 'tpch-sample-data' jars from jars/3rdparty folder.
Aditya Kishore [Sun, 11 Sep 2016 07:19:25 +0000 (00:19 -0700)] 
Exclude 'drill-memory-base' and 'tpch-sample-data' jars from jars/3rdparty folder.

5 years agoExplicitly specify `hbase-server` dependency in Hive storage plugin.
Aditya Kishore [Fri, 9 Sep 2016 22:53:06 +0000 (15:53 -0700)] 
Explicitly specify `hbase-server` dependency in Hive storage plugin.

Hive's HBaseStorageHandler uses HBase's TableInputFormat which is in hbase-server module.

5 years agoUpdated plugin version to 1.9.0-SNAPSHOT
Aditya [Thu, 8 Sep 2016 22:30:40 +0000 (15:30 -0700)] 
Updated plugin version to 1.9.0-SNAPSHOT

5 years agoDRILL-4199: Add Support for HBase 1.X
Aditya [Thu, 31 Mar 2016 00:55:59 +0000 (17:55 -0700)] 
DRILL-4199: Add Support for HBase 1.X

5 years agoUpdate MapR v5.1.0 artifacts version
Patrick Wong [Thu, 24 Mar 2016 23:36:38 +0000 (16:36 -0700)] 
Update MapR v5.1.0 artifacts version

5 years agoMD-813: Improve count(*) queries against MapR-DB Json tables.
Aditya [Wed, 16 Mar 2016 08:58:56 +0000 (01:58 -0700)] 
MD-813: Improve count(*) queries against MapR-DB Json tables.

+ Fail query on schema change.
+ Added a configuration option 'ignoreSchemaChange', which when enabled, drops the rows from the result

5 years agoMD-789: Query with condition involving addition of DATE and INTERVAL types returns...
Aditya [Fri, 11 Mar 2016 10:23:35 +0000 (02:23 -0800)] 
MD-789: Query with condition involving addition of DATE and INTERVAL types returns no results [MapR-DB JSON Tables]

+ Added `enablePushdown` option to enable/disable all filter pushdown, enabled by default.

5 years agoUpdated plugin version to 1.7.0-SNAPSHOT
Patrick Wong [Fri, 11 Mar 2016 01:37:34 +0000 (17:37 -0800)] 
Updated plugin version to 1.7.0-SNAPSHOT

5 years agoMD-572: Column names in MapR-DB JSON tables are case-sensitive
Aditya [Wed, 9 Mar 2016 00:57:52 +0000 (16:57 -0800)] 
MD-572: Column names in MapR-DB JSON tables are case-sensitive

Disable pushdown of both filter and projects (by default set to false).

This will allow Drill to handle both of these operators in a case-insensitive way.

5 years agoRefactoring code for better organization.
Aditya [Fri, 4 Mar 2016 23:17:36 +0000 (15:17 -0800)] 
Refactoring code for better organization.

+ Adding skeleton streams plugin.

5 years agoMD-771: Query with nested functions within a sub-query on MapR-DB JSON Tables fail...
Aditya [Tue, 8 Mar 2016 09:40:39 +0000 (01:40 -0800)] 
MD-771: Query with nested functions within a sub-query on MapR-DB JSON Tables fail with ClassCastException

Fix for failing test cases.

5 years agoMD-726: Add support for `read_numbers_as_double` and `all_text_mode` options
Smidth Panchamia [Thu, 3 Mar 2016 06:32:37 +0000 (22:32 -0800)] 
MD-726: Add support for `read_numbers_as_double` and `all_text_mode` options

+ MD-773: Add support to push-down filters on Date, Time and Timestamp
+ Fix the offsets in Date and Time data types in returned values
+ Updated support for DATE, TIME, and TIMESTAMP types
+ Modified unit-tests to check for operation push-down

5 years agoMD-711: Enable support for UNION types in Drill-MapRDB plugin
Smidth Panchamia [Thu, 25 Feb 2016 16:46:38 +0000 (16:46 +0000)] 
MD-711: Enable support for UNION types in Drill-MapRDB plugin

5 years agoMD-741: Cannot select array elements by index in MapR-DB JSON Tables
Aditya [Thu, 25 Feb 2016 10:34:30 +0000 (02:34 -0800)] 
MD-741: Cannot select array elements by index in MapR-DB JSON Tables

5 years agoMD-548: Remove reference to mapr-release pom.
Aditya [Wed, 24 Feb 2016 07:15:36 +0000 (23:15 -0800)] 
MD-548: Remove reference to mapr-release pom.

* Bumped the Drill+plugin version to 1.6.0-SNAPSHOT.
* Updated MapR client artifacts to 5.1.0.
* Added missing Apache license headers to few files.
* Added a base test class BaseJsonTest for all OJAI Unit tests.
* Invoke GuavaPatcher from BaseJsonTest.

5 years agoMD-735: Handle CastExpression in filter push-down.
Smidth Panchamia [Tue, 23 Feb 2016 01:09:53 +0000 (17:09 -0800)] 
MD-735: Handle CastExpression in filter push-down.

5 years agoSupport for `_id` only projection.
Smidth Panchamia [Fri, 19 Feb 2016 18:41:56 +0000 (18:41 +0000)] 
Support for `_id` only projection.

5 years agoMD-719: Fix creating QueryConditions for JsonSubScanSpec
Smidth Panchamia [Wed, 17 Feb 2016 23:51:26 +0000 (23:51 +0000)] 
MD-719: Fix creating QueryConditions for JsonSubScanSpec

5 years agoMD-703: Cannot select `_id` field from MapR-DB JSON Table
Aditya [Fri, 5 Feb 2016 00:43:27 +0000 (16:43 -0800)] 
MD-703: Cannot select `_id` field from MapR-DB JSON Table

+ Included Smidth's fix for column shuffling.
+ Fix the private-drill-mapr-plugin build.

5 years agoEnhance exception handling.
Aditya [Thu, 4 Feb 2016 01:46:44 +0000 (17:46 -0800)] 
Enhance exception handling.

5 years agoMAPR-22405: DocumentStream interface in OJAI API need not be parametrized
Aditya [Tue, 2 Feb 2016 10:04:22 +0000 (02:04 -0800)] 
MAPR-22405: DocumentStream interface in OJAI API need not be parametrized

5 years agoMD-670: Querying MapR-DB JSON Tables returns no results
Aditya [Tue, 2 Feb 2016 01:43:42 +0000 (17:43 -0800)] 
MD-670: Querying MapR-DB JSON Tables returns no results

* Use DocumentReader API to emit "_id" field instead of handling it as a special case.
* Update the DrillBuf reference field when reallocation happen.
* Catch the correct exception when schema change happens and include the field name in the warning message.
+ Get rid of unused code.

5 years agoUpdated with the OJAI APIs changes in Date/Time/Document classes.
Aditya [Mon, 4 Jan 2016 05:23:06 +0000 (21:23 -0800)] 
Updated with the OJAI APIs changes in Date/Time/Document classes.

5 years agoMD-578: Query with three-way join fails with IllegalStateException
Smidth Panchamia [Fri, 8 Jan 2016 18:00:57 +0000 (10:00 -0800)] 
MD-578: Query with three-way join fails with IllegalStateException

5 years agoUpdate the json-support branch version in pom.xml
Smidth Panchamia [Tue, 5 Jan 2016 00:30:43 +0000 (16:30 -0800)] 
Update the json-support branch version in pom.xml

+ Fix for column name mangling in drill output.
+ Comment out Date, Time, Timestamp types since we are still deciding these on OJAI front.

5 years agoUpdating with the new DocumentReader APIs.
Aditya [Mon, 21 Dec 2015 08:50:19 +0000 (00:50 -0800)] 
Updating with the new DocumentReader APIs.

5 years agoFilter push-down support for JSON tables.
Smidth Panchamia [Sat, 24 Oct 2015 22:48:35 +0000 (15:48 -0700)] 
Filter push-down support for JSON tables.

5 years agoUse DBDocumentReaderBase instead of DBDocumentReader.
Aditya Kishore [Fri, 26 Aug 2016 20:12:41 +0000 (13:12 -0700)] 
Use DBDocumentReaderBase instead of DBDocumentReader.

5 years agoAdding support for Json tables.
Aditya [Thu, 15 Oct 2015 06:49:09 +0000 (23:49 -0700)] 
Adding support for Json tables.

+ Re-factored code to separate binary table specific code from the common code.
+ Added test cases

5 years agoCleaning up code.
Aditya [Fri, 17 Jul 2015 00:59:52 +0000 (17:59 -0700)] 
Cleaning up code.

Removed some unnecessary files from the module.

5 years agoFix the private-drill-mapr-plugin build.
Smidth Panchamia [Wed, 10 Feb 2016 17:57:37 +0000 (17:57 +0000)] 
Fix the private-drill-mapr-plugin build.

5 years agoUpdated plugin version to 1.5.0-SNAPSHOT
Patrick Wong [Tue, 8 Dec 2015 19:25:16 +0000 (11:25 -0800)] 
Updated plugin version to 1.5.0-SNAPSHOT

5 years agoUpdating the plugin version to 1.4.0-SNAPSHOT and fixing for interface change.
Aditya [Thu, 3 Dec 2015 19:54:17 +0000 (11:54 -0800)] 
Updating the plugin version to 1.4.0-SNAPSHOT and fixing for interface change.

5 years agoApplying DRILL-4020 (The not-equal operator returns incorrect results when used on...
Aditya [Fri, 6 Nov 2015 23:56:36 +0000 (15:56 -0800)] 
Applying DRILL-4020 (The not-equal operator returns incorrect results when used on the HBase row key)

5 years agoApplying DRILL-3621 (Wrong results when Drill on HBase query contains rowkey "or...
Aditya [Fri, 6 Nov 2015 23:45:21 +0000 (15:45 -0800)] 
Applying DRILL-3621 (Wrong results when Drill on HBase query contains rowkey "or" or "IN") to MapRDB plugin

Already in json_support branch, exclude while merging.

5 years agoMD-524: Drill API change broke the MapRDB Plugin
Aditya [Fri, 6 Nov 2015 01:09:51 +0000 (17:09 -0800)] 
MD-524: Drill API change broke the MapRDB Plugin