drill.git
6 years agoTag release drill-1.2.0 1.2.0 drill-1.2.0
adeneche [Mon, 12 Oct 2015 02:56:00 +0000 (19:56 -0700)] 
Tag release drill-1.2.0

6 years agoDRILL-3918: During expansion save the metadata for future use. 196/head
Aman Sinha [Sun, 11 Oct 2015 15:02:30 +0000 (08:02 -0700)] 
DRILL-3918: During expansion save the metadata for future use.

close apache/drill#196

6 years agoDRILL-3917: During file selection expansion, get the metadata for the directory by...
Aman Sinha [Sat, 10 Oct 2015 09:18:40 +0000 (02:18 -0700)] 
DRILL-3917: During file selection expansion, get the metadata for the directory by reading the metadata file.
Ensure the selection root format is the same as the format of the files.
Add unit test.

close apache/drill#195

6 years agoDRILL-3916: Add JDBC plugin to assembly 192/head
aleph-zero [Thu, 8 Oct 2015 19:25:11 +0000 (12:25 -0700)] 
DRILL-3916: Add JDBC plugin to assembly

This commits adds the JDBC plugin jar to the assembly so that it can be
loaded by Drill as a storage plugin.

6 years agoDRILL-3791: Fix bugs in JDBC storage plugin
aleph-zero [Thu, 1 Oct 2015 20:23:41 +0000 (13:23 -0700)] 
DRILL-3791: Fix bugs in JDBC storage plugin

Fixes issues with bit, date, time and timestamp types in MySQL.

6 years agoDRILL-3888: Build test jars for all Drill Modules
Aditya Kishore [Fri, 2 Oct 2015 18:36:45 +0000 (11:36 -0700)] 
DRILL-3888: Build test jars for all Drill Modules

Move the test jar configuration to the root pom and remove it from individual module's pom.

This closes #188

6 years agoDRILL-3901: Don't do early expansion of directory in the non-metadata-cache case...
Aman Sinha [Wed, 7 Oct 2015 02:38:04 +0000 (19:38 -0700)] 
DRILL-3901:  Don't do early expansion of directory in the non-metadata-cache case because it already happens during ParquetGroupScan's metadata gathering operation.

Update unit test that was changed by a previous fix for DRILL-3788

6 years agoDRILL-3892: Once usedMetadataFile is set to true, don't change it.
Aman Sinha [Sun, 4 Oct 2015 23:48:58 +0000 (16:48 -0700)] 
DRILL-3892: Once usedMetadataFile is set to true, don't change it.
Modify unit test to test partition pruning with metadata cache.
Fix indentation for a comment.

6 years agoupgading maven-release plugin to fix release issues
adeneche [Mon, 5 Oct 2015 01:04:59 +0000 (18:04 -0700)] 
upgading maven-release plugin to fix release issues

6 years agoadding gpg signing key for adeneche
adeneche [Sat, 3 Oct 2015 03:23:51 +0000 (20:23 -0700)] 
adding gpg signing key for adeneche

6 years agoDRILL-2879: Part 2 - Enhancing extended json support for date in millies and binary...
Kamesh [Wed, 15 Jul 2015 11:04:41 +0000 (16:34 +0530)] 
DRILL-2879: Part 2 - Enhancing extended json support for date in millies and binary with type info

Addressing review comments

Updated unit test to remove timezone that was being pulled from the local system
(and thus failed to match the baseline if run from a different timezone)

6 years agoDRILL-3887: Fix bug where metadata file not being used
Steven Phillips [Thu, 1 Oct 2015 23:54:18 +0000 (16:54 -0700)] 
DRILL-3887: Fix bug where metadata file not being used

6 years agoDRILL-3874: flattening large JSON objects uses too much direct memory - add getBuffer...
Chris Westin [Tue, 29 Sep 2015 23:05:00 +0000 (16:05 -0700)] 
DRILL-3874: flattening large JSON objects uses too much direct memory - add getBufferSizeFor() to ValueVector interface - add implememtations of getBufferSizeFor() for all ValueVector derivatives - add adaptive algorithm for adjusting batch size to flatten operator

6 years agoDRILL-2361: Allow column aliases to include dots.
AdamPD [Fri, 2 Oct 2015 00:54:04 +0000 (17:54 -0700)] 
DRILL-2361: Allow column aliases to include dots.

6 years agoDRILL-3884: Fix lower parallelization issues with Hive's native scan. 185/head
vkorukanti [Thu, 1 Oct 2015 22:15:12 +0000 (15:15 -0700)] 
DRILL-3884: Fix lower parallelization issues with Hive's native scan.

This closes #185

6 years agoDRILL-1065: Support for ALTER ... RESET statement
Sudheesh Katkam [Tue, 15 Sep 2015 23:13:23 +0000 (16:13 -0700)] 
DRILL-1065: Support for ALTER ... RESET statement

+ Support for "SET option = value" statement (assumes scope as SESSION)
+ Bump Calcite version to include CALCITE-823 (Parser support for "ALTER
  ... RESET" statement). This commit includes a breaking change:
  SqlSetOption#getName now returns a SqlIdentifier rather than a String
  => option names are multi-part identifiers, and do not
  require escaping

+ Add rule in CompoundIdentifierConverter (+ Override annotations)
+ Improve error messages in SetOptionHandler
+ Add documentation (CompoundIdentifierConverter, OptionValue,
  SessionOptionManager, SystemOptionManager)
- Does not include support for deleting short lived options

+ Default ExecutionControls option value should be at SYSTEM level
+ Change asserts to preconditions in SystemOptionManager
+ Add a precondition to TypeValidator's ctor to ensure default value are
  set at SYSTEM level

this closes #159

6 years agoDRILL-3788: part 2: When building the metadata file path for the single entry case...
Aman Sinha [Wed, 30 Sep 2015 08:43:02 +0000 (01:43 -0700)] 
DRILL-3788: part 2: When building the metadata file path for the single entry case, check if the entry is a directory.

Close apache/drill#179

6 years agoDRILL-3788: Expand the file selection to contain all files within the directory while...
Mehant Baid [Thu, 24 Sep 2015 23:35:47 +0000 (16:35 -0700)] 
DRILL-3788: Expand the file selection to contain all files within the directory while creating DynamicDrillTable

6 years agoDRILL-3819: Remove redundant check to ignore files beginning with '.'
Mehant Baid [Tue, 22 Sep 2015 20:13:22 +0000 (13:13 -0700)] 
DRILL-3819: Remove redundant check to ignore files beginning with '.'

6 years agoDRILL-3817: Disable rewriting compound identifier for refresh metadata query
Mehant Baid [Tue, 22 Sep 2015 20:09:51 +0000 (13:09 -0700)] 
DRILL-3817: Disable rewriting compound identifier for refresh metadata query

6 years agoDRILL-3869: Allow queries submitted through the web UI to end with a semi-colon
aleph-zero [Thu, 1 Oct 2015 18:54:09 +0000 (11:54 -0700)] 
DRILL-3869: Allow queries submitted through the web UI to end with a semi-colon

This closes #178

6 years agoDRILL-3479: Fix sqlline version for all profiles
Parth Chandra [Wed, 30 Sep 2015 20:06:16 +0000 (13:06 -0700)] 
DRILL-3479: Fix sqlline version for all profiles

6 years agoDRILL-3725: Add HTTPS support for Drill web interface
vkorukanti [Mon, 31 Aug 2015 20:13:05 +0000 (13:13 -0700)] 
DRILL-3725: Add HTTPS support for Drill web interface

6 years agoDRILL-3257: Disable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule and...
Hsuan-Yi Chu [Sun, 27 Sep 2015 00:16:20 +0000 (17:16 -0700)] 
DRILL-3257: Disable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule and add test cases

Close apache/drill#176

6 years agoDRILL-3209: Add support for reading Hive parquet tables using Drill native parquet...
vkorukanti [Fri, 25 Sep 2015 17:52:08 +0000 (10:52 -0700)] 
DRILL-3209: Add support for reading Hive parquet tables using Drill native parquet reader

6 years agoDRILL-3479: Sqlline shows incorrect version of Drill
Parth Chandra [Wed, 23 Sep 2015 23:47:44 +0000 (16:47 -0700)] 
DRILL-3479: Sqlline shows incorrect version of Drill

6 years agoDRILL-2274: Unable to allocate sv2 buffer after repeated attempts : JOIN, Order by...
adeneche [Mon, 21 Sep 2015 18:59:56 +0000 (11:59 -0700)] 
DRILL-2274: Unable to allocate sv2 buffer after repeated attempts : JOIN, Order by used in query

- when ExternalSortBatch.newSV2() couldn't allocate a new SV2 it spills the spilledBatchGroups instead of batchGroups, otherwise no memory will be released for the sort's allocator
- copierAllocator is closed as soon as possible to free more memory for the sort

this closes #172

6 years agoDRILL-2583, DRILL-3428: Catch exceptions, and throw UserException#dataReadError with...
Sudheesh Katkam [Thu, 17 Sep 2015 23:48:31 +0000 (16:48 -0700)] 
DRILL-2583, DRILL-3428: Catch exceptions, and throw UserException#dataReadError with more context. This closes #161

+ Added convenient method to UserException for String.format(...)

6 years agoDRILL-2908: Fix Parquet for var length vectors where encoding changes across pages...
Parth Chandra [Tue, 15 Sep 2015 22:27:17 +0000 (15:27 -0700)] 
DRILL-2908: Fix Parquet for var length vectors where encoding changes across pages. Add unit tests. Add option to make parquet page size and disctionary page size configurable at session level. This closes #162

6 years agoDRILL-2908:Enable reading the Int 96 type from parquet files.
Jason Altekruse [Tue, 14 Apr 2015 23:27:59 +0000 (16:27 -0700)] 
DRILL-2908:Enable reading the Int 96 type from parquet files.

column chunk metadata can be out of order from the column ordering in the schema, even though it exposes both as a list, making them seem like they should correspond, so we have to make our own map between the column names and indexes in the list.

Support for varbinary reading and int96 reading in the new reader.

Support the second version page header, the java library will only dictionary encode fix length byte arrays when the writer version is set to 2.0

Looks to be working in the vectorized reader, need a test case.

Fixed complex reader, was using the wrong field to figure out the length to read.

Conflicts:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReaders.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetToDrillTypeConverter.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetGroupConverter.java

UDF for reading impala timestamps from varbinary

Fix for reading fixed binary and int96 columns in the vectorized parquet reader.

Conflicts:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java

Fix for a bug reading fixed binary and int 96 data out of parquet when the data is plain encoded.

6 years agoDRILL-3596: Allow only (<expression>) or (<expression>, 1) for LEAD and LAG window...
adeneche [Mon, 24 Aug 2015 20:37:25 +0000 (13:37 -0700)] 
DRILL-3596: Allow only (<expression>) or (<expression>, 1) for LEAD and LAG window functions as input parameters

this closes #128

6 years agoDRILL-3822: Have PathScanner use own, not thread-context, class loader.
dbarclay [Wed, 23 Sep 2015 01:04:45 +0000 (18:04 -0700)] 
DRILL-3822:  Have PathScanner use own, not thread-context, class loader.

this closes #166

6 years agoDRILL-3784: simple Jdbc program fails with NoClassDefFoundError
adeneche [Wed, 23 Sep 2015 03:34:19 +0000 (20:34 -0700)] 
DRILL-3784: simple Jdbc program fails with NoClassDefFoundError

6 years agoDRILL-1457: Push Limit past through UnionExchange.
Jinfeng Ni [Tue, 22 Sep 2015 05:42:23 +0000 (22:42 -0700)] 
DRILL-1457: Push Limit past through UnionExchange.

Close apache/drill#169

6 years agoDRILL-3836: Find free port automatically in TestBitRpc
Julien Le Dem [Thu, 24 Sep 2015 20:52:08 +0000 (13:52 -0700)] 
DRILL-3836: Find free port automatically in TestBitRpc

This closes #168

6 years agoDRILL-3778: Add missed part of DRILL-3160 (making JDBC Javadoc available).
dbarclay [Mon, 14 Sep 2015 17:54:34 +0000 (10:54 -0700)] 
DRILL-3778: Add missed part of DRILL-3160 (making JDBC Javadoc available).

This closes #158

Main:
Configured Javadoc generation (title, package groups, version in headers).
Added link to JDBC page in Drill documentation site.
Edited/fixed some JDBC Javadoc comments.
Added explicit SQLConversionOverflowException to throws clauses for Javadoc
effect.
Added some imports for Javadoc references.

Misc.:
Fixed a couple Javadoc syntax errors.
Fixed POM indentation.

6 years agoDRILL-3781: Group by system function in schema-based table.
Jinfeng Ni [Mon, 21 Sep 2015 21:59:03 +0000 (14:59 -0700)] 
DRILL-3781:  Group by system function in schema-based table.

Fix is in Calcite-886. Add unit test and bump forked Calcite version in Drill.

6 years agoDRILL-3811: AtomicRemainder incorrectly accounts for transferred allocations
adeneche [Sun, 20 Sep 2015 19:11:55 +0000 (12:11 -0700)] 
DRILL-3811: AtomicRemainder incorrectly accounts for transferred allocations

this closes #163

6 years agoDRILL-2748: Improve cost estimation for Drill logical aggregation in query planner.
Jinfeng Ni [Wed, 16 Sep 2015 05:05:21 +0000 (22:05 -0700)] 
DRILL-2748: Improve cost estimation for Drill logical aggregation in query planner.

6 years agoDRILL-3735: For partition pruning divide up the partition lists into sublists of...
Aman Sinha [Sat, 12 Sep 2015 19:57:12 +0000 (12:57 -0700)] 
DRILL-3735: For partition pruning divide up the partition lists into sublists of 64K each and iterate over each sublist.

Add abstract base class for various partition descriptors.  Add logging messages in PruneScanRule for better debuggability.

Address review comments.

Close apache/drill#156

6 years agoDRILL-3732: Drill leaks memory if external sort hits out of disk space exception
adeneche [Sat, 5 Sep 2015 00:10:17 +0000 (17:10 -0700)] 
DRILL-3732: Drill leaks memory if external sort hits out of disk space exception

- ExternalSort.mergeAndSpill() cleans all it's data in case an errors occurs while it's spilling to disk
- made BatchGroup AutoCloseable so it can easily be closed with AutoCloseables.close() if an error occurs
- added injection site while External sort is spilling to disk
- added unit test that forces a 2 batch query to spill to disk and injects an exception while it does so

this closes #147

6 years agoDRILL-3809: Make PrelFinalizer have instances rather than a single Prel since RelShut...
Jacques Nadeau [Sun, 20 Sep 2015 00:40:05 +0000 (17:40 -0700)] 
DRILL-3809: Make PrelFinalizer have instances rather than a single Prel since RelShuttleImpl has internal state.

6 years agoreverting a fix no longer needed after DRILL-3767
adeneche [Fri, 11 Sep 2015 21:51:52 +0000 (14:51 -0700)] 
reverting a fix no longer needed after DRILL-3767

6 years agoDRILL-2743: Parquet file metadata caching
Steven Phillips [Tue, 18 Aug 2015 20:24:33 +0000 (13:24 -0700)] 
DRILL-2743: Parquet file metadata caching

rebasing on top of master required conflict resolution in Parser.tdd and parserImpls.ftl

this closes #114

6 years agoDRILL-3767: SchemaPath.getCompoundPath(String...strings) reverses it's input array
adeneche [Fri, 11 Sep 2015 17:31:59 +0000 (10:31 -0700)] 
DRILL-3767: SchemaPath.getCompoundPath(String...strings) reverses it's input array

this closes #155

6 years agoDRILL-3779: Fix NPE in mergeAndSpill(). Add more debug logging messages. Make copie...
Aman Sinha [Thu, 17 Sep 2015 08:29:38 +0000 (01:29 -0700)] 
DRILL-3779: Fix NPE in mergeAndSpill().  Add more debug logging messages.  Make copier per-batch memory limit power of 2.
Get rid of some warnings.  Add a few comments.

Address review comments.

Close apache/drill#160

6 years agoDRILL-1942-readers:
Chris Westin [Fri, 11 Sep 2015 01:28:00 +0000 (18:28 -0700)] 
DRILL-1942-readers:
- add extends AutoCloseable to RecordReader, and rename cleanup() to close().
- fix many warnings
- formatting fixes

DRILL-1942-readers:
- renamed cleanup() to close in the new JdbcRecordReader

Close apache/drill#154

6 years agoDRILL-1942-concurrency-test: new smoke test for concurrent query execution; useful...
Chris Westin [Fri, 7 Aug 2015 00:09:26 +0000 (17:09 -0700)] 
DRILL-1942-concurrency-test: new smoke test for concurrent query execution; useful to checking on the new allocator's locking schemes, as well as a general concurrent query execution smoke test. this closes #105

6 years agoDRILL-3680: Fix incorrect query result or IOBE when window function is used in subquery.
Jinfeng Ni [Tue, 25 Aug 2015 17:56:25 +0000 (10:56 -0700)] 
DRILL-3680: Fix incorrect query result or IOBE when window function is used in subquery.

6 years agoDRILL-3761: Modify CastIntDecimal implementation so that it will not modify its input...
Jinfeng Ni [Fri, 28 Aug 2015 23:50:21 +0000 (16:50 -0700)] 
DRILL-3761: Modify CastIntDecimal implementation so that it will not modify its input holder.

6 years agoDRILL-3718: After TextReader finishes reading a field surrounded by double quotes...
Hsuan-Yi Chu [Fri, 28 Aug 2015 00:02:10 +0000 (17:02 -0700)] 
DRILL-3718: After TextReader finishes reading a field surrounded by double quotes, the reader would skip whitespaces only if those whitespaces are not used as delimiter

6 years agoDRILL-3724: Improve javadoc of the plugin component
Edmon Begoli [Sun, 30 Aug 2015 14:56:22 +0000 (10:56 -0400)] 
DRILL-3724: Improve javadoc of the plugin component

Improve the javadoc of the key methods and classes for custom plugin development.

this closes #139

6 years agoDRILL-3347: VARCHAR ResultSet.getObject returned ...hadoop.io.Text, not String.
dbarclay [Tue, 4 Aug 2015 23:51:07 +0000 (16:51 -0700)] 
DRILL-3347: VARCHAR ResultSet.getObject returned ...hadoop.io.Text, not String.

this closes #144
Core fix:
- Fixed {,Nullable}VarCharAccessor's getObject() to return String instead of
  value vector's internal org.apache.hadoop.io.Text.
- Updated unit tests (to expect only String now).
  [DatabaseMetaDataGetColumnsTest, ResultSetMetaDataTest]

Also Added getObject check in tracing proxy test.  [TracingProxyDriverTest]
Changed hard references to Hadoop's Text and JodaTime's Period to strings in
warning check in tracing proxy.  [InvocationReporterImpl]

Cleanup:
- Added @Override annotations.  [SqlAccessors]
- (Unintentionally) fixed (undetected) missing comma.  [ValueVectorTypes.tdd]

6 years agoDRILL-1942-hygiene:
Chris Westin [Tue, 25 Aug 2015 23:45:35 +0000 (16:45 -0700)] 
DRILL-1942-hygiene:
- add AutoCloseable to many classes
- minor fixes
- formatting

this closes #133

6 years agoDRILL-2650: Mark query end time when closing the Foreman
Sudheesh Katkam [Thu, 23 Jul 2015 00:21:50 +0000 (17:21 -0700)] 
DRILL-2650: Mark query end time when closing the Foreman

this closes #80

6 years agoDRILL-3160: Make JDBC Javadoc documentation available to users
Aditya Kishore [Mon, 14 Sep 2015 02:44:48 +0000 (19:44 -0700)] 
DRILL-3160: Make JDBC Javadoc documentation available to users

6 years agoDelete deprecated Jackson access proxies.
Jacques Nadeau [Mon, 14 Sep 2015 02:40:30 +0000 (19:40 -0700)] 
Delete deprecated Jackson access proxies.

6 years agoDRILL-3589: Update JDBC driver to shade and minimize dependencies.
Jacques Nadeau [Sun, 16 Aug 2015 18:46:26 +0000 (11:46 -0700)] 
DRILL-3589: Update JDBC driver to shade and minimize dependencies.

Update build process to use shading, stop using Proguard.
Add simple integration test that verifies that the JDBC driver works
correctly with a clean classpath.

6 years agoDisable TestExampleQueries.testTextPartitions() until DRILL-3774 is fixed.
Jacques Nadeau [Mon, 14 Sep 2015 02:14:26 +0000 (19:14 -0700)] 
Disable TestExampleQueries.testTextPartitions() until DRILL-3774 is fixed.

6 years agoReduce test time by not waiting 2 seconds for newly submitted tasks before shutting...
Jacques Nadeau [Mon, 14 Sep 2015 01:51:46 +0000 (18:51 -0700)] 
Reduce test time by not waiting 2 seconds for newly submitted tasks before shutting down RPC thread pool.

6 years agoDRILL-3458: Enhancing Avro file format by support nested data types like union, map
Kamesh [Sat, 4 Jul 2015 18:05:48 +0000 (23:35 +0530)] 
DRILL-3458: Enhancing Avro file format by support nested data types like union, map

Removing unused imports and fixing typos in the comments

Adding sanity check to verify whether union is an optional field or not and negative testcases

Handling string and utf-8 datatypes properly

removing bracket notation and documentation

6 years agoDRILL-3720: Avro Record Reader should process Avro files by per block basis
Kamesh [Fri, 28 Aug 2015 09:50:44 +0000 (15:20 +0530)] 
DRILL-3720: Avro Record Reader should process Avro files by per block basis

6 years agoDRILL-2879: Enhancing extended json support for date in millies and binary with type...
Kamesh [Wed, 15 Jul 2015 11:04:41 +0000 (16:34 +0530)] 
DRILL-2879: Enhancing extended json support for date in millies and binary with type info

Ignore project push down Mongo test until test completes correctly on Linux

6 years agoDRILL-3773: Fix Mongo FieldSelection
Jason Altekruse [Wed, 9 Sep 2015 18:26:09 +0000 (11:26 -0700)] 
DRILL-3773: Fix Mongo FieldSelection

Mongo plugin was previously rewriting a complex (multi-level) column reference as a simple selection of the top level field.

This changeset does not change this behavior in terms of the filter sent to mongo, but it add the original selected column to the list that will be read in by the JSON reader once that data is returned from mongo.

What this means is that we will be requesting more data from mongo that necessary (as we were previously), but this will be leveraging the existing functionality in the JSON reader to grab only the sub-selection actually requested in the query. This allows for difficult schema changes to be avoided by projecting only columns without schema changes.

This also fixes and adds unit tests for FieldSelection that cause wrong results when selecting a nested column and its parent.

6 years agoDRILL-1666: Add test code for mongo storage plugin
akumarb2010 [Mon, 6 Jul 2015 17:05:25 +0000 (22:35 +0530)] 
DRILL-1666: Add test code for mongo storage plugin

6 years agoDRILL-3180: JDBC Storage Plugin updates. 115/head
Jacques Nadeau [Sun, 2 Aug 2015 01:11:51 +0000 (18:11 -0700)] 
DRILL-3180: JDBC Storage Plugin updates.

- Move to leverage Calcite's JDBC adapter capabilities for pushdowns, schema, etc.
- Add test cases using Derby

6 years agoDRILL-3180: Initial JDBC plugin implementation.
MPierre [Sun, 2 Aug 2015 01:07:18 +0000 (18:07 -0700)] 
DRILL-3180: Initial JDBC plugin implementation.

6 years agoDRILL-3280, DRILL-3360, DRILL-3601, DRILL-3649: Add test cases
Hsuan-Yi Chu [Wed, 9 Sep 2015 23:21:54 +0000 (16:21 -0700)] 
DRILL-3280, DRILL-3360, DRILL-3601, DRILL-3649: Add test cases

Fix is in CALCITE-820

Close apache/drill#152

6 years agoDRILL-3755: In DrillSqlWorker, give UserException.validationError if ValidationExcept...
Hsuan-Yi Chu [Mon, 7 Sep 2015 23:47:18 +0000 (16:47 -0700)] 
DRILL-3755: In DrillSqlWorker, give UserException.validationError if ValidationException is thrown from Calcite

6 years agoDRILL-2190, DRILL-2313, DRILL-2318: Add test cases
Hsuan-Yi Chu [Fri, 4 Sep 2015 18:35:45 +0000 (11:35 -0700)] 
DRILL-2190, DRILL-2313, DRILL-2318: Add test cases

Fixes are in CALCITE-634, CALCITE-613, CALCITE-662

6 years agoDRILL-3683: Add baseline and expected plan for TestWindowFunctions suite
Hsuan-Yi Chu [Sat, 22 Aug 2015 00:11:49 +0000 (17:11 -0700)] 
DRILL-3683: Add baseline and expected plan for TestWindowFunctions suite

6 years agoDRILL-3412: Add ProjectWindowTransposeRule to push Project past Window
Hsuan-Yi Chu [Fri, 21 Aug 2015 05:33:11 +0000 (22:33 -0700)] 
DRILL-3412: Add ProjectWindowTransposeRule to push Project past Window

Fix is in CALCITE-844

6 years agoDRILL-3580: Add test case and bump calcite version to 1.4.0-drill-r2
Hsuan-Yi Chu [Tue, 8 Sep 2015 17:42:51 +0000 (10:42 -0700)] 
DRILL-3580: Add test case and bump calcite version to 1.4.0-drill-r2

Fix is in CALCITE-841

6 years agoDRILL-1942-hygiene
Chris Westin [Mon, 17 Aug 2015 22:42:28 +0000 (15:42 -0700)] 
DRILL-1942-hygiene

- Formatting
- @Overrides
- finals
- some AutoCloseable additions
- new isCancelled() abstract method on FragmentManager, implemented on subclasses

Added missing new abstract method isCancelled()

Close apache/drill#120

6 years agoDRILL-3746: Get Hive partition values from MetaStore instead of from parsing the... 151/head
vkorukanti [Thu, 10 Sep 2015 00:42:45 +0000 (17:42 -0700)] 
DRILL-3746: Get Hive partition values from MetaStore instead of from parsing the partition location path

1) Added a partition with custom location to test Hive table. Existing partition tests now work after the fix.
2) Enabled a test which was disabled previously due to a bug in interpreter code which is fixed recently.

This closes #151

6 years agoDRILL-3467: Restrict visibily of workspaces in 'show schemas' based on access permiss...
vkorukanti [Tue, 8 Sep 2015 07:39:38 +0000 (00:39 -0700)] 
DRILL-3467: Restrict visibily of workspaces in 'show schemas' based on access permissions of querying user.

Also:
Fixed failing tests in TestJdbcMetadata as the result depends on whether /tmp/drill_test is available on
test system or not. Change it to use the default connection test parameters (which creates a unique tmp directory)
so that the test result doesn't depend on individual machine.

6 years agoDRILL-3535: Add support for Drop Table
Mehant Baid [Wed, 5 Aug 2015 19:03:01 +0000 (12:03 -0700)] 
DRILL-3535: Add support for Drop Table

this closes #140

6 years agoDRILL-3622: When user authentication is enabled, enforce admin privileges to update...
vkorukanti [Tue, 8 Sep 2015 17:01:14 +0000 (17:01 +0000)] 
DRILL-3622: When user authentication is enabled, enforce admin privileges to update SYSTEM options

+ define what user is considered an admin
+ remove a stray file in test module (exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/RootResource.java)

6 years agoDRILL-3668: Incorrect results FIRST_VALUE function
adeneche [Wed, 19 Aug 2015 18:52:29 +0000 (11:52 -0700)] 
DRILL-3668: Incorrect results FIRST_VALUE function

added DefaultFrameTemplate.resetInternal() and generate code to set first value of internal batch to NULL at the end of each partition
added unit test to make sure bug has been fixed

this closes #146

6 years agoDRILL-3566: Fix: PreparedStatement.executeQuery() got ClassCastException.
dbarclay [Tue, 28 Jul 2015 02:27:50 +0000 (19:27 -0700)] 
DRILL-3566: Fix:  PreparedStatement.executeQuery() got ClassCastException.

Main:
Restored DrillResultSetImpl(...)'s statement parameter from overly
restrictive DrillStatementImpl to AvaticaStatement and removed caller
cast that was throwing.  (Relatedly, adjusted getStatement() and moved
internal casting from statement to connection.)

Added basic test of querying via PreparedStatement.  [PreparedStatementTest]
Added some case test of statement-creation methods.  [ConnectionTest]

Removed now-unneeded DrillPrepareResult cast.
Eliminated redundant getConnection() and getClient calls, intermediate variables, etc.
Fixed comment error.

this closes #143

6 years agoDRILL-3555: Changing defaults for planner.memory.max_query_memory_per_node causes...
adeneche [Thu, 27 Aug 2015 19:57:23 +0000 (12:57 -0700)] 
DRILL-3555: Changing defaults for planner.memory.max_query_memory_per_node causes queries with window function to fail

this closes #137

6 years agoDRILL-3497: Throw UserException#validationError instead of SetOptionException and...
Sudheesh Katkam [Tue, 14 Jul 2015 22:33:30 +0000 (15:33 -0700)] 
DRILL-3497: Throw UserException#validationError instead of SetOptionException and ExpressionParsingException for options

This closes #98

+ Fixed bug in PositiveLongValidator (0 is not positive)
+ Added UserExceptionMatcher for unit tests

DRILL-3497: Addressing Jason's review comments (1)

6 years agoDRILL-3455: If fragments on unregistered Drillbits finished successfully, do not...
Sudheesh Katkam [Thu, 23 Jul 2015 00:16:29 +0000 (17:16 -0700)] 
DRILL-3455: If fragments on unregistered Drillbits finished successfully, do not fail the query

+ DRILL-3448: Flipped the atLeastOneFailure condition in QueryManager
+ fixes in DrillbitStatusListener interface
+ logs from implementations of DrillbitStatusListener

6 years agoDRILL-2304: Manage options with lowercase names
Sudheesh Katkam [Wed, 22 Jul 2015 23:32:02 +0000 (16:32 -0700)] 
DRILL-2304: Manage options with lowercase names

This closes #90

Major refactoring:
+ Removed unnecessary methods from OptionManagers, and refactored implementations
+ Removed OptionAdmin interface, which was a delegate to validators

Other fixes:
+ Added CaseInsensitiveMap
+ Moved OptionValue creation from TypeValidators to SetOptionHandler
+ Fixed error message in BaseOptionManager
+ Added documentation for all OptionManagers, and OptionValidator

DRILL-2304: Addressing Jason's review comments (1)

+ Use generic Map rather than CaseInsensitiveMap for member variables and parameters
+ SessionOptionManager should use a ConcurrentHashMap

6 years agoDRILL-3711: Fix unit tests on Windows
Parth Chandra [Wed, 26 Aug 2015 01:04:11 +0000 (18:04 -0700)] 
DRILL-3711: Fix unit tests on Windows

6 years agoDRILL-3492: Add support for encoding/decoding of to/from OrderedBytes
spanchamia [Thu, 30 Jul 2015 05:53:04 +0000 (22:53 -0700)] 
DRILL-3492: Add support for encoding/decoding of to/from OrderedBytes
format

Description:
This change allows encoding/decoding of data from/to 'double', 'float',
'bigint', 'int' and 'utf8' data types to/from OrderedBytes format.
It also allows for OrderedByte encoded row-keys to be stored in
ascending as well as descending order.

The following JIRA added the OrderedBytes encoding to HBase:
https://issues.apache.org/jira/browse/HBASE-8201

This encoding scheme will preserve the sort-order of the native
data-type when it is stored as sorted byte arrays on disk.
Thus, it will help the HBase storage plugin if the row-keys have been
encoded in OrderedBytes format.

This functionality allows us to prune the scan ranges, thus reading much
lesser data from the server.

Testing Done:
Added a new unit-test class TestOrderedBytesConvertFunctions.java which
derives from TestConvertFunctions.java class.
Also add new test cases to TestHBaseFilterPushDown class that will test
if we were able to push-down filters correctly and if the results are
correct.

DRILL-3492 - * Remove repeated allocations of byte arrays and PositionedByteRange objects
on heap(as suggested by Jason).
* Remove OrderedBytes encode/decode operations on UTF8 types.
Reasons -
1. These operations are slow and incur a lot of heap allocations
2. UTF8 types maintain their natural sort order when stored as binary arrays.

DRILL-3492 - Remove test code that creates test tables with UTF8 OrderedByte encoding.

6 years agoDRILL-3669: Fix missing indirect dependency.
Julien Le Dem [Wed, 19 Aug 2015 05:24:52 +0000 (22:24 -0700)] 
DRILL-3669: Fix missing indirect dependency.

6 years agoDRILL-3598: use a factory to create the root allocator.
Chris Westin [Tue, 4 Aug 2015 13:26:57 +0000 (06:26 -0700)] 
DRILL-3598: use a factory to create the root allocator.
- made the constructor for TopLevelAllocator package private to enforce this

Delete a test that had been commented out for over a year, it no longer compiles due to interface changes and there is plenty of other testing for hash aggregate.

6 years agoDRILL-3661: Update: Added a @link and a @code.
dbarclay [Wed, 2 Sep 2015 17:14:37 +0000 (10:14 -0700)] 
DRILL-3661: Update:  Added a @link and a @code.

This closes #119

6 years agoDRILL-3661: Update: Fixed "DRILL-xxxx" reference.
dbarclay [Tue, 1 Sep 2015 16:56:17 +0000 (09:56 -0700)] 
DRILL-3661: Update:  Fixed "DRILL-xxxx" reference.

6 years agoDRILL-3661: Update: Fixed bad import.
dbarclay [Thu, 20 Aug 2015 05:42:09 +0000 (22:42 -0700)] 
DRILL-3661: Update:  Fixed bad import.

6 years agoDRILL-3661: Edit JDBC doc. for clarity, more consistency, bug fixes.
dbarclay [Tue, 18 Aug 2015 00:26:45 +0000 (17:26 -0700)] 
DRILL-3661: Edit JDBC doc. for clarity, more consistency, bug fixes.

Also fixed two serialVersionUID literals.

6 years agoDRILL-3707: Fix for DRILL-3616 can cause a NullPointerException in ExternalSort cleanup
adeneche [Tue, 25 Aug 2015 19:25:24 +0000 (12:25 -0700)] 
DRILL-3707: Fix for DRILL-3616 can cause a NullPointerException in ExternalSort cleanup

This closes #130

6 years agoDRILL-3684: CTAS : Memory Leak when using CTAS with tpch sf100
adeneche [Mon, 31 Aug 2015 22:09:29 +0000 (15:09 -0700)] 
DRILL-3684: CTAS : Memory Leak when using CTAS with tpch sf100

This closes #141

6 years agoDRILL-3719: Expand -expression as -1 * expression in visitCall of DrillOptiq
Hsuan-Yi Chu [Fri, 28 Aug 2015 17:33:01 +0000 (10:33 -0700)] 
DRILL-3719: Expand -expression as -1 * expression in visitCall of DrillOptiq

6 years agoDRILL-3045: Make planning time memory limit configurable
Mehant Baid [Fri, 28 Aug 2015 18:23:54 +0000 (11:23 -0700)] 
DRILL-3045: Make planning time memory limit configurable

6 years agoDRILL-2748: Add optimizer rule to push filter past aggregate.
Jinfeng Ni [Thu, 27 Aug 2015 21:54:28 +0000 (14:54 -0700)] 
DRILL-2748: Add optimizer rule to push filter past aggregate.

6 years agoDRILL-3313: Address review comments
Parth Chandra [Tue, 25 Aug 2015 18:39:21 +0000 (11:39 -0700)] 
DRILL-3313: Address review comments

6 years agoDRILL-3313: Eliminate redundant #load methods and unit-test loading & exporting of...
Hanifi Gunes [Tue, 23 Jun 2015 19:05:45 +0000 (12:05 -0700)] 
DRILL-3313: Eliminate redundant #load methods and unit-test loading & exporting of vectors

6 years agoDRILL-3153: Fix JDBC's getIdentifierQuoteString() to report Drill's backquote.
dbarclay [Fri, 24 Jul 2015 00:46:05 +0000 (17:46 -0700)] 
DRILL-3153: Fix JDBC's getIdentifierQuoteString() to report Drill's backquote.

Added override of Avatica's default implementation returning SQL
standard value.  Added Javadoc.  Added unit test.

Also moved unit test for nullsAreSortedXxx methods.

DRILL-3153: Update:  Comment fixes and whitespace only.

Close apache/drill#99