6 years ago[maven-release-plugin] prepare release drill-1.3.0 1.3.0 drill-1.3.0
Jacques Nadeau [Wed, 18 Nov 2015 06:05:19 +0000 (22:05 -0800)] 
[maven-release-plugin] prepare release drill-1.3.0

6 years agoDRILL-4056: Fix corruption bug reading string data out of Avro
Jason Altekruse [Fri, 13 Nov 2015 23:46:58 +0000 (15:46 -0800)] 
DRILL-4056: Fix corruption bug reading string data out of Avro

- Fix issue where we are reading a byte array without considering length
- Removed use of unnecessary Holder objects.
- Added restriction on batch size produced by a single call to next.
- Add some basic result verification to avro tests.

6 years agoDRILL-4063: Add s3a dependency jars and example core-site.xml
Abhi P [Wed, 18 Nov 2015 03:04:23 +0000 (19:04 -0800)] 
DRILL-4063: Add s3a dependency jars and example core-site.xml

6 years agoDRILL-4103: add drill.version to parquet metadata
Julien Le Dem [Wed, 18 Nov 2015 01:19:35 +0000 (17:19 -0800)] 
DRILL-4103: add drill.version to parquet metadata

6 years agoDRILL-4041 & DRILL-4057: Disable RPC thread offload until concurrency bug is found.
Jacques Nadeau [Fri, 6 Nov 2015 22:25:05 +0000 (14:25 -0800)] 
DRILL-4041 & DRILL-4057: Disable RPC thread offload until concurrency bug is found.

6 years agoDRILL-4065: Maintain a cache of created UserGroupInformation objects so that we don...
Jacques Nadeau [Wed, 11 Nov 2015 00:04:09 +0000 (16:04 -0800)] 
DRILL-4065: Maintain a cache of created UserGroupInformation objects so that we don't accidentally generate thousands of FileSystem objects & threads.

6 years agoAdd synchronization to DrillBuf.retain() to avoid contention issues.
Jacques Nadeau [Sat, 7 Nov 2015 03:09:19 +0000 (19:09 -0800)] 
Add synchronization to DrillBuf.retain() to avoid contention issues.

6 years agoAdd assertion around QueryDataBatch release to ensure that we're not releasing the...
Jacques Nadeau [Sat, 7 Nov 2015 03:08:30 +0000 (19:08 -0800)] 
Add assertion around QueryDataBatch release to ensure that we're not releasing the buffers twice.

6 years agoDRILL-4048: Fix reading required dictionary encoded varbinary data in parquet files...
Jason Altekruse [Sat, 7 Nov 2015 03:24:28 +0000 (19:24 -0800)] 
DRILL-4048: Fix reading required dictionary encoded varbinary data in parquet files after recent update

Fix was small, this update is a little larger than necessary because I was hoping to create
a unit test by modifying the one I had added in the earlier patch with the version upgrade.
Unfortunately we don't have a good way to generate Parquet files with required columns from
unit tests right now. So I just added a smaller subset of the binary file that was posted on
the JIRA issue. The refactoring of the earlier test was still useful for readability,
so I kept it in.

6 years agoDRILL-4049: Terminate StatusThread when Workmanager is closed.
Jacques Nadeau [Sat, 7 Nov 2015 02:58:01 +0000 (18:58 -0800)] 
DRILL-4049: Terminate StatusThread when Workmanager is closed.

Avoid excessive growth in threads with long-lived test JVMs.

6 years agoDRILL-4046: Upgrade Joda to later version (avoid contention issues)
Jacques Nadeau [Sat, 7 Nov 2015 02:57:00 +0000 (18:57 -0800)] 
DRILL-4046: Upgrade Joda to later version (avoid contention issues)

6 years agoDRILL-4042: Unable to run sqlline in embedded mode on Windows
Patrick Wong [Fri, 6 Nov 2015 21:05:09 +0000 (13:05 -0800)] 
DRILL-4042: Unable to run sqlline in embedded mode on Windows

Use newer version of hadoop-winutils

6 years agoDRILL-4025: Create FileSelection object with the appropriate file statuses
Mehant Baid [Mon, 2 Nov 2015 22:26:43 +0000 (14:26 -0800)] 
DRILL-4025: Create FileSelection object with the appropriate file statuses

6 years agoDRILL-4040: Build failure on master 244/head
Aditya Kishore [Thu, 5 Nov 2015 21:03:45 +0000 (13:03 -0800)] 
DRILL-4040: Build failure on master

This closes #244

6 years agoDRILL-4006: Reallocate offset vector in repeated vectors when index is beyond the...
Hsuan-Yi Chu [Sun, 1 Nov 2015 00:14:34 +0000 (17:14 -0700)] 
DRILL-4006: Reallocate offset vector in repeated vectors when index is beyond the current capacity

Author:     Steven Phillips <smp@apache.org>

This closes #243, #242

6 years agoDRILL-3994: make classpath scanning work on windows
Julien Le Dem [Fri, 30 Oct 2015 23:51:09 +0000 (16:51 -0700)] 
DRILL-3994: make classpath scanning work on windows

6 years agoDRILL-3793: New MergeJoin and add RecordIterator interface
Amit Hadke [Thu, 8 Oct 2015 01:23:21 +0000 (18:23 -0700)] 
DRILL-3793: New MergeJoin and add RecordIterator interface

This closes #190

6 years agoDRILL-4028: Update Drill to leverage latest version of Parquet library.
Jason Altekruse [Fri, 4 Sep 2015 18:09:23 +0000 (11:09 -0700)] 
DRILL-4028: Update Drill to leverage latest version of Parquet library.

- Remove references to the shaded version of a Jackson @JsonCreator annotation from parquet, replace with proper fasterxml version.
- Fixing imports using the wrong parquet packages after rebase.
- Fixing issues with Drill parquet read a write path after merging the Drill parquet fork back into mainline.
- Fixed the issue with the writer, needed to flush the RecordConsumer in the ParquetRecordWriter.
- Consolidate page reading code
- Added some test to print out some additional context when an ordered comparison of two datasets fails in a test.
- Fix up parquet API usage in Hive Module.
- Adding unit test to read a write all types in parquet, the decimal types and interval year have some issues.
- Use direct codec factory from new package in the parquet library now that it has been moved.
- Moving the test for Direct Codec Factory out of the Drill source as the class itself has been moved.
- Small fix after consolidating two different ByteBuffer based implementations of BytesInput.
- Small fixes to accommodate interface changes.
- Small changes to remove direct references to DirectCodecFactory, this class is not accessible outside of parquet, but an instance with the same contract is now accessible with a new factory method on CodecFactory.
- Fixed failing test using miniDFS when reading a larger parquet file.

This closes #236

6 years agoDRILL-4031: Log warning and ignore columns returned from JDBC source that are unsuppo...
Jacques Nadeau [Wed, 4 Nov 2015 18:46:36 +0000 (10:46 -0800)] 
DRILL-4031: Log warning and ignore columns returned from JDBC source that are unsupported.

This closes #240

6 years agoDRILL-951: Add support for csv header row parsing
Abhi P [Tue, 3 Nov 2015 03:03:04 +0000 (19:03 -0800)] 
DRILL-951: Add support for csv header row parsing

This closes #232

6 years agoDRILL-3963: Add Sequence file support.
Amit Hadke [Wed, 21 Oct 2015 20:55:40 +0000 (13:55 -0700)] 
DRILL-3963: Add Sequence file support.

- File Support
- Add test for sequencefile reader in Impersonation test suite.
- Create hadoop based record reader under user proxy.
- Fix impersonation test for sequence file and add one for Avro.

This closes #214

6 years agoDRILL-3952: Improve Window Functions performance when not all batches are required...
adeneche [Thu, 22 Oct 2015 21:08:10 +0000 (14:08 -0700)] 
DRILL-3952: Improve Window Functions performance when not all batches are required to process the current batch

this closes #222

6 years agoDRILL-4032: Check if a new vector was created to determine if allocation is necessary...
Steven Phillips [Wed, 4 Nov 2015 22:17:39 +0000 (14:17 -0800)] 
DRILL-4032: Check if a new vector was created to determine if allocation is necessary in MapWriter
closes #241

6 years agoDRILL-3634: Add number of partitions information in the plan for hive scans
Mehant Baid [Thu, 29 Oct 2015 22:10:12 +0000 (15:10 -0700)] 
DRILL-3634: Add number of partitions information in the plan for hive scans

closes #231

6 years agoDRILL-3941: Add timing instrumentation in different phases of partition pruning
Mehant Baid [Fri, 23 Oct 2015 21:01:09 +0000 (14:01 -0700)] 
DRILL-3941: Add timing instrumentation in different phases of partition pruning
Add basic timing instrumentation while initializing FileSelection and reading the metadata cache file

closes #230

6 years agoDRILL-3912: Common subexpression elimination
Steven Phillips [Wed, 7 Oct 2015 10:44:10 +0000 (03:44 -0700)] 
DRILL-3912: Common subexpression elimination

Closes #189

6 years agoDRILL-3229: Miscellaneous Union-type fixes
Steven Phillips [Thu, 22 Oct 2015 00:00:44 +0000 (17:00 -0700)] 
DRILL-3229: Miscellaneous Union-type fixes

closes #207
closes #180

6 years agoDRILL-3232: Promotable writer
Steven Phillips [Fri, 9 Oct 2015 19:59:37 +0000 (12:59 -0700)] 
DRILL-3232: Promotable writer

6 years agoDRILL-3233: Expression handling for Union types
Steven Phillips [Mon, 5 Oct 2015 04:32:34 +0000 (21:32 -0700)] 
DRILL-3233: Expression handling for Union types

6 years agoDRILL-3229: Implement Union type vector
Steven Phillips [Thu, 1 Oct 2015 10:26:34 +0000 (03:26 -0700)] 
DRILL-3229: Implement Union type vector

6 years agoDRILL-3871: Off by one error while reading binary fields with one terminal null in... 219/head
Parth Chandra [Mon, 5 Oct 2015 17:25:56 +0000 (10:25 -0700)] 
DRILL-3871: Off by one error while reading binary fields with one terminal null in parquet.

6 years agoDRILL-3242: Update RPC layer so that requests and response are managed on a secondary... 229/head
Jacques Nadeau [Mon, 25 May 2015 05:01:49 +0000 (22:01 -0700)] 
DRILL-3242: Update RPC layer so that requests and response are managed on a secondary thread.
- Create a separate serialized executor for fragment receiverFinished events.
- Update serialized executor to pool object creation.
- Ensure that FragmentExecutor acceptExternalEvents countdown occurs when only execution is cancellation.

6 years agoSerializedExecutor sketch for consideration.
Chris Westin [Fri, 22 May 2015 21:17:41 +0000 (14:17 -0700)] 
SerializedExecutor sketch for consideration.

Based on the EventProcessor, but recast as an Executor that accepts Runnables.
The idea would be to use this on RPC reception points to immediately hand off
work. Each Fragment would have a SerializedExecutor associated with it, serving
as a queue to deliver events to when the RPC thread receives them.

6 years agoDRILL-3810: Add Avro schema support
Kamesh [Mon, 2 Nov 2015 01:15:01 +0000 (17:15 -0800)] 
DRILL-3810: Add Avro schema support

Also updates the FileFormatPlugin to support providing schema.

6 years agoDRILL-1752: Fixing query failures of unsharded db/collection in sharded cluster
Kamesh [Tue, 20 Oct 2015 05:56:04 +0000 (11:26 +0530)] 
DRILL-1752: Fixing query failures of unsharded db/collection in sharded cluster

- Add logback.xml in test/resources
- Add testcases and upgrading pom version of mongo and mongo test dependency

6 years agoDRILL-3921: Initialize the underlying record reader lazily in HiveRecordReader
Sudheesh Katkam [Mon, 26 Oct 2015 20:18:35 +0000 (13:18 -0700)] 
DRILL-3921: Initialize the underlying record reader lazily in HiveRecordReader

This closes #197

+ OperatorContextImpl has a lazily initialized executor service that is
  a decorator around the worker pool. This service is used through
  #runCallableAs to run a callable task as the given proxy user.
+ Unit tests use partition student table with chained impersonation to
  exercise running multiple delegate callables (creates multiple readers,
  a reader per partition).

+ Remove unused WorkManager#shutdownAndAwaitTermination method

6 years agoDRILL-3983: Small test improvements improve error message when SQL parsing error...
Julien Le Dem [Mon, 26 Oct 2015 21:13:58 +0000 (14:13 -0700)] 
DRILL-3983: Small test improvements improve error message when SQL parsing error add a simple test to Parquet writer make errors verbose by default in tests

This closes #221

Also includes:
 - make the checkstyle conf not tied to the execution to allow running mvn checkstyle:check from the command line
 - fix verbose output in tests

6 years agoDRILL-3956: Add support for MySQL text type
Jacques Nadeau [Mon, 2 Nov 2015 00:14:21 +0000 (16:14 -0800)] 
DRILL-3956: Add support for MySQL text type

6 years agoDRILL-3992: Add/fix support for JDBC schemas (tested against oracle and derby)
Jacques Nadeau [Thu, 29 Oct 2015 00:20:51 +0000 (17:20 -0700)] 
DRILL-3992: Add/fix support for JDBC schemas (tested against oracle and derby)

This closes #225

6 years agoDRILL-4000: Ensure storage plugins are not needlessly created. Add start and close...
Jacques Nadeau [Sat, 31 Oct 2015 22:16:32 +0000 (15:16 -0700)] 
DRILL-4000: Ensure storage plugins are not needlessly created. Add start and close capability to storage plugins.

This closes #227

Add a new configuration based map so any configurations that are passed to a node can leverage existing storage plugins.
Update FileSystemConfig to correctly implement hashcode()
Update StoragePlugin interface to extends Autocloseable and add start() method.
Update Mongo plugin to close clients when closing plugin.

6 years agoDRILL-3937: Handle the case where min/max columns in metadata cache file are string... 220/head
Aman Sinha [Mon, 26 Oct 2015 07:02:11 +0000 (00:02 -0700)] 
DRILL-3937: Handle the case where min/max columns in metadata cache file are string or binary values.

Simplify serialization and check for nulls. Remove byte array comparison for min/max since it is not needed anymore.

close apache/drill#220

6 years agoDRILL-2967: Incompatible types error reported in a "not in" query with compatible...
adeneche [Mon, 19 Oct 2015 22:40:05 +0000 (15:40 -0700)] 
DRILL-2967: Incompatible types error reported in a "not in" query with compatible data types

6 years agoDRILL-3980: Upgrade hadoop dependency version to 2.7.x in mapr profile
vkorukanti [Fri, 23 Oct 2015 07:06:16 +0000 (00:06 -0700)] 
DRILL-3980: Upgrade hadoop dependency version to 2.7.x in mapr profile

6 years agoDRILL-3749: Upgrade Hadoop to 2.7.1
aleph-zero [Wed, 14 Oct 2015 21:31:57 +0000 (14:31 -0700)] 
DRILL-3749: Upgrade Hadoop to 2.7.1

Upgrades Hadoop dependency from 2.4.1 to 2.7.1. This allows us to take
advantage of the latest S3 filesystem support. Includes update to Hadoop
methods and updated curator library (2.7.1)

Also includes adding @Ignore to TestDrillbitResilience.cancelAfterEverythingIsCompleted() as it appears to be subject to a timing error.

See: DRILL-3967

This closes #203

6 years agoDRILL-3742: Classpath scanning and build improvement
Julien Le Dem [Thu, 15 Oct 2015 17:59:18 +0000 (10:59 -0700)] 
DRILL-3742: Classpath scanning and build improvement

Makes the classpath scanning a build time class discovery
Makes the fmpp generation incremental
Removes some slowness in DrillBit closing
Reduces the build time by 30%

This closes #148

6 years agoDRILL-3914: Gis contrib module with basic spatial queries functionality
potocki [Thu, 8 Oct 2015 06:10:33 +0000 (08:10 +0200)] 
DRILL-3914: Gis contrib module with basic spatial queries functionality

This closes #191

6 years agoDRILL-3505: Add the _id when using a * projection in Mongo
Tugdual Grall [Tue, 20 Oct 2015 09:32:37 +0000 (11:32 +0200)] 
DRILL-3505: Add the _id when using a * projection in Mongo

This closes #210

6 years agoDRILL-2726: Add Drill version, from Manifest, same as SqlLine in the
Tugdual Grall [Tue, 13 Oct 2015 12:45:57 +0000 (05:45 -0700)] 
DRILL-2726: Add Drill version, from Manifest, same as SqlLine in the

This closes #198

6 years agoDRILL-3975: Make sure to strip scheme and authority from partition location.
Jacques Nadeau [Sun, 25 Oct 2015 15:59:39 +0000 (08:59 -0700)] 
DRILL-3975: Make sure to strip scheme and authority from partition location.

This closes #217

6 years agoDRILL-3340: Part 2: Reverting 1a589ab and committing latest patch
Sudheesh Katkam [Mon, 26 Oct 2015 03:24:46 +0000 (20:24 -0700)] 
DRILL-3340: Part 2: Reverting 1a589ab and committing latest patch

Add operator metrics registry for metric definitions
+ Display metrics as a table within an operator profile panel
+ Rename FragmentStats#getOperatorStats to newOperatorStats

6 years agoDRILL-3340: Added operator names and metric names to query profile before writing...
Sudheesh Katkam [Mon, 13 Jul 2015 21:19:57 +0000 (14:19 -0700)] 
DRILL-3340: Added operator names and metric names to query profile before writing it to store

+ Rename: FragmentStats#getOperatorStats => newOperatorStats
+ Documentation

this closes #216

6 years agoDRILL-3965: Fix IOOB while trying to perform partition pruning
Mehant Baid [Tue, 20 Oct 2015 06:49:45 +0000 (23:49 -0700)] 
DRILL-3965: Fix IOOB while trying to perform partition pruning

6 years agoDRILL-3769: Allow to_date() to use castDate()'s implementations
Hsuan-Yi Chu [Tue, 13 Oct 2015 05:52:05 +0000 (22:52 -0700)] 
DRILL-3769: Allow to_date() to use castDate()'s implementations

6 years agoDRILL-3429: Cast input of statistical window functions to double before rewriting...
Mehant Baid [Thu, 8 Oct 2015 20:47:59 +0000 (13:47 -0700)] 
DRILL-3429: Cast input of statistical window functions to double before rewriting the expression.

6 years agoDRILL-3802: Throw unsupported error for ROLLUP, CUBE, GROUPING SETS, GROUPING, GROUPI... 209/head
Jinfeng Ni [Mon, 19 Oct 2015 17:19:03 +0000 (10:19 -0700)] 

close apache/drill#209

6 years agomove version to 1.3.0-SNAPSHOT
adeneche [Mon, 12 Oct 2015 02:56:00 +0000 (19:56 -0700)] 
move version to 1.3.0-SNAPSHOT

6 years agoDRILL-3947: Use setSafe() for date, time, timestamp types while populating pruning... 208/head
Aman Sinha [Sun, 18 Oct 2015 16:59:19 +0000 (09:59 -0700)] 
DRILL-3947: Use setSafe() for date, time, timestamp types while populating pruning vector (other types were already using setSafe).

close apache/drill#208

6 years agoDRILL-3943: In constant folding, when the the data type of the return value is TimeSt...
Hsuan-Yi Chu [Thu, 15 Oct 2015 03:55:34 +0000 (20:55 -0700)] 
DRILL-3943: In constant folding, when the the data type of the return value is TimeStamp, generate TimeStamp Literal

This closes #206

6 years agoDRILL-3930: Remove direct references to TopLevelAllocator from unit tests
Chris Westin [Tue, 13 Oct 2015 20:49:57 +0000 (13:49 -0700)] 
DRILL-3930: Remove direct references to TopLevelAllocator from unit tests
Ensure RootAllocatorFactory is used throughout the code so
that we can change allocators via configuration or software. Use
DrillAutoCloseables to handle exceptions that could happen from some
allocator close() calls when the allocator is in an improper state.

this closes #200

6 years agoDRILL-3927: use OutOfMemoryException in more places
Chris Westin [Fri, 9 Oct 2015 21:08:09 +0000 (14:08 -0700)] 
DRILL-3927: use OutOfMemoryException in more places
Some code cleanup required for the upcoming introduction of the rewritten
direct memory allocator. Chiefly the introduction of OutOfMemoryException
handling in a few spots, the use of DrillAutoCloseables for some allocator
close() calls, and some other minor cleanup.
- removed the exclusion of exec/work from the jdbc-all jar, because it now
  depends on OutOfMemoryException, and that is derived from
  FragmentSetupException and Foreman Exception

this closes #199

6 years agoDRILL-3876: Skip SplitUpComplexExpression logic only when there is one complex expres...
Jinfeng Ni [Tue, 13 Oct 2015 00:23:48 +0000 (17:23 -0700)] 
DRILL-3876: Skip SplitUpComplexExpression logic only when there is one complex expression at tope level of expression tree.

6 years agoDRILL-3876: Avoid an extra copy of the original list when flattening
Jason Altekruse [Tue, 6 Oct 2015 16:23:18 +0000 (09:23 -0700)] 
DRILL-3876: Avoid an extra copy of the original list when flattening

This only fixes a basic case, a more complete refactoring of the rewrite rule could avoid copies in cases with multiple flattens, this will be addressed in DRILL-3899.

close apache/drill#187

6 years agoDRILL-3920: Additional tests added to TestValueVectors for serialization and loading.
Chris Westin [Fri, 9 Oct 2015 01:15:57 +0000 (18:15 -0700)] 
DRILL-3920: Additional tests added to TestValueVectors for serialization and loading.
Some light cleanup of a few vector implementations.

closes #194

6 years agoDRILL-3918: During expansion save the metadata for future use. 196/head
Aman Sinha [Sun, 11 Oct 2015 15:02:30 +0000 (08:02 -0700)] 
DRILL-3918: During expansion save the metadata for future use.

close apache/drill#196

6 years agoDRILL-3917: During file selection expansion, get the metadata for the directory by...
Aman Sinha [Sat, 10 Oct 2015 09:18:40 +0000 (02:18 -0700)] 
DRILL-3917: During file selection expansion, get the metadata for the directory by reading the metadata file.
Ensure the selection root format is the same as the format of the files.
Add unit test.

close apache/drill#195

6 years agoDRILL-3916: Add JDBC plugin to assembly 192/head
aleph-zero [Thu, 8 Oct 2015 19:25:11 +0000 (12:25 -0700)] 
DRILL-3916: Add JDBC plugin to assembly

This commits adds the JDBC plugin jar to the assembly so that it can be
loaded by Drill as a storage plugin.

6 years agoDRILL-3791: Fix bugs in JDBC storage plugin
aleph-zero [Thu, 1 Oct 2015 20:23:41 +0000 (13:23 -0700)] 
DRILL-3791: Fix bugs in JDBC storage plugin

Fixes issues with bit, date, time and timestamp types in MySQL.

6 years agoDRILL-3888: Build test jars for all Drill Modules
Aditya Kishore [Fri, 2 Oct 2015 18:36:45 +0000 (11:36 -0700)] 
DRILL-3888: Build test jars for all Drill Modules

Move the test jar configuration to the root pom and remove it from individual module's pom.

This closes #188

6 years agoDRILL-3901: Don't do early expansion of directory in the non-metadata-cache case...
Aman Sinha [Wed, 7 Oct 2015 02:38:04 +0000 (19:38 -0700)] 
DRILL-3901:  Don't do early expansion of directory in the non-metadata-cache case because it already happens during ParquetGroupScan's metadata gathering operation.

Update unit test that was changed by a previous fix for DRILL-3788

6 years agoDRILL-3892: Once usedMetadataFile is set to true, don't change it.
Aman Sinha [Sun, 4 Oct 2015 23:48:58 +0000 (16:48 -0700)] 
DRILL-3892: Once usedMetadataFile is set to true, don't change it.
Modify unit test to test partition pruning with metadata cache.
Fix indentation for a comment.

6 years agoupgading maven-release plugin to fix release issues
adeneche [Mon, 5 Oct 2015 01:04:59 +0000 (18:04 -0700)] 
upgading maven-release plugin to fix release issues

6 years agoadding gpg signing key for adeneche
adeneche [Sat, 3 Oct 2015 03:23:51 +0000 (20:23 -0700)] 
adding gpg signing key for adeneche

6 years agoDRILL-2879: Part 2 - Enhancing extended json support for date in millies and binary...
Kamesh [Wed, 15 Jul 2015 11:04:41 +0000 (16:34 +0530)] 
DRILL-2879: Part 2 - Enhancing extended json support for date in millies and binary with type info

Addressing review comments

Updated unit test to remove timezone that was being pulled from the local system
(and thus failed to match the baseline if run from a different timezone)

6 years agoDRILL-3887: Fix bug where metadata file not being used
Steven Phillips [Thu, 1 Oct 2015 23:54:18 +0000 (16:54 -0700)] 
DRILL-3887: Fix bug where metadata file not being used

6 years agoDRILL-3874: flattening large JSON objects uses too much direct memory - add getBuffer...
Chris Westin [Tue, 29 Sep 2015 23:05:00 +0000 (16:05 -0700)] 
DRILL-3874: flattening large JSON objects uses too much direct memory - add getBufferSizeFor() to ValueVector interface - add implememtations of getBufferSizeFor() for all ValueVector derivatives - add adaptive algorithm for adjusting batch size to flatten operator

6 years agoDRILL-2361: Allow column aliases to include dots.
AdamPD [Fri, 2 Oct 2015 00:54:04 +0000 (17:54 -0700)] 
DRILL-2361: Allow column aliases to include dots.

6 years agoDRILL-3884: Fix lower parallelization issues with Hive's native scan. 185/head
vkorukanti [Thu, 1 Oct 2015 22:15:12 +0000 (15:15 -0700)] 
DRILL-3884: Fix lower parallelization issues with Hive's native scan.

This closes #185

6 years agoDRILL-1065: Support for ALTER ... RESET statement
Sudheesh Katkam [Tue, 15 Sep 2015 23:13:23 +0000 (16:13 -0700)] 
DRILL-1065: Support for ALTER ... RESET statement

+ Support for "SET option = value" statement (assumes scope as SESSION)
+ Bump Calcite version to include CALCITE-823 (Parser support for "ALTER
  ... RESET" statement). This commit includes a breaking change:
  SqlSetOption#getName now returns a SqlIdentifier rather than a String
  => option names are multi-part identifiers, and do not
  require escaping

+ Add rule in CompoundIdentifierConverter (+ Override annotations)
+ Improve error messages in SetOptionHandler
+ Add documentation (CompoundIdentifierConverter, OptionValue,
  SessionOptionManager, SystemOptionManager)
- Does not include support for deleting short lived options

+ Default ExecutionControls option value should be at SYSTEM level
+ Change asserts to preconditions in SystemOptionManager
+ Add a precondition to TypeValidator's ctor to ensure default value are
  set at SYSTEM level

this closes #159

6 years agoDRILL-3788: part 2: When building the metadata file path for the single entry case...
Aman Sinha [Wed, 30 Sep 2015 08:43:02 +0000 (01:43 -0700)] 
DRILL-3788: part 2: When building the metadata file path for the single entry case, check if the entry is a directory.

Close apache/drill#179

6 years agoDRILL-3788: Expand the file selection to contain all files within the directory while...
Mehant Baid [Thu, 24 Sep 2015 23:35:47 +0000 (16:35 -0700)] 
DRILL-3788: Expand the file selection to contain all files within the directory while creating DynamicDrillTable

6 years agoDRILL-3819: Remove redundant check to ignore files beginning with '.'
Mehant Baid [Tue, 22 Sep 2015 20:13:22 +0000 (13:13 -0700)] 
DRILL-3819: Remove redundant check to ignore files beginning with '.'

6 years agoDRILL-3817: Disable rewriting compound identifier for refresh metadata query
Mehant Baid [Tue, 22 Sep 2015 20:09:51 +0000 (13:09 -0700)] 
DRILL-3817: Disable rewriting compound identifier for refresh metadata query

6 years agoDRILL-3869: Allow queries submitted through the web UI to end with a semi-colon
aleph-zero [Thu, 1 Oct 2015 18:54:09 +0000 (11:54 -0700)] 
DRILL-3869: Allow queries submitted through the web UI to end with a semi-colon

This closes #178

6 years agoDRILL-3479: Fix sqlline version for all profiles
Parth Chandra [Wed, 30 Sep 2015 20:06:16 +0000 (13:06 -0700)] 
DRILL-3479: Fix sqlline version for all profiles

6 years agoDRILL-3725: Add HTTPS support for Drill web interface
vkorukanti [Mon, 31 Aug 2015 20:13:05 +0000 (13:13 -0700)] 
DRILL-3725: Add HTTPS support for Drill web interface

6 years agoDRILL-3257: Disable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule and...
Hsuan-Yi Chu [Sun, 27 Sep 2015 00:16:20 +0000 (17:16 -0700)] 
DRILL-3257: Disable FilterSetOpTransposeRule, DrillProjectSetOpTransposeRule and add test cases

Close apache/drill#176

6 years agoDRILL-3209: Add support for reading Hive parquet tables using Drill native parquet...
vkorukanti [Fri, 25 Sep 2015 17:52:08 +0000 (10:52 -0700)] 
DRILL-3209: Add support for reading Hive parquet tables using Drill native parquet reader

6 years agoDRILL-3479: Sqlline shows incorrect version of Drill
Parth Chandra [Wed, 23 Sep 2015 23:47:44 +0000 (16:47 -0700)] 
DRILL-3479: Sqlline shows incorrect version of Drill

6 years agoDRILL-2274: Unable to allocate sv2 buffer after repeated attempts : JOIN, Order by...
adeneche [Mon, 21 Sep 2015 18:59:56 +0000 (11:59 -0700)] 
DRILL-2274: Unable to allocate sv2 buffer after repeated attempts : JOIN, Order by used in query

- when ExternalSortBatch.newSV2() couldn't allocate a new SV2 it spills the spilledBatchGroups instead of batchGroups, otherwise no memory will be released for the sort's allocator
- copierAllocator is closed as soon as possible to free more memory for the sort

this closes #172

6 years agoDRILL-2583, DRILL-3428: Catch exceptions, and throw UserException#dataReadError with...
Sudheesh Katkam [Thu, 17 Sep 2015 23:48:31 +0000 (16:48 -0700)] 
DRILL-2583, DRILL-3428: Catch exceptions, and throw UserException#dataReadError with more context. This closes #161

+ Added convenient method to UserException for String.format(...)

6 years agoDRILL-2908: Fix Parquet for var length vectors where encoding changes across pages...
Parth Chandra [Tue, 15 Sep 2015 22:27:17 +0000 (15:27 -0700)] 
DRILL-2908: Fix Parquet for var length vectors where encoding changes across pages. Add unit tests. Add option to make parquet page size and disctionary page size configurable at session level. This closes #162

6 years agoDRILL-2908:Enable reading the Int 96 type from parquet files.
Jason Altekruse [Tue, 14 Apr 2015 23:27:59 +0000 (16:27 -0700)] 
DRILL-2908:Enable reading the Int 96 type from parquet files.

column chunk metadata can be out of order from the column ordering in the schema, even though it exposes both as a list, making them seem like they should correspond, so we have to make our own map between the column names and indexes in the list.

Support for varbinary reading and int96 reading in the new reader.

Support the second version page header, the java library will only dictionary encode fix length byte arrays when the writer version is set to 2.0

Looks to be working in the vectorized reader, need a test case.

Fixed complex reader, was using the wrong field to figure out the length to read.


UDF for reading impala timestamps from varbinary

Fix for reading fixed binary and int96 columns in the vectorized parquet reader.


Fix for a bug reading fixed binary and int 96 data out of parquet when the data is plain encoded.

6 years agoDRILL-3596: Allow only (<expression>) or (<expression>, 1) for LEAD and LAG window...
adeneche [Mon, 24 Aug 2015 20:37:25 +0000 (13:37 -0700)] 
DRILL-3596: Allow only (<expression>) or (<expression>, 1) for LEAD and LAG window functions as input parameters

this closes #128

6 years agoDRILL-3822: Have PathScanner use own, not thread-context, class loader.
dbarclay [Wed, 23 Sep 2015 01:04:45 +0000 (18:04 -0700)] 
DRILL-3822:  Have PathScanner use own, not thread-context, class loader.

this closes #166

6 years agoDRILL-3784: simple Jdbc program fails with NoClassDefFoundError
adeneche [Wed, 23 Sep 2015 03:34:19 +0000 (20:34 -0700)] 
DRILL-3784: simple Jdbc program fails with NoClassDefFoundError

7 years agoDRILL-1457: Push Limit past through UnionExchange.
Jinfeng Ni [Tue, 22 Sep 2015 05:42:23 +0000 (22:42 -0700)] 
DRILL-1457: Push Limit past through UnionExchange.

Close apache/drill#169

7 years agoDRILL-3836: Find free port automatically in TestBitRpc
Julien Le Dem [Thu, 24 Sep 2015 20:52:08 +0000 (13:52 -0700)] 
DRILL-3836: Find free port automatically in TestBitRpc

This closes #168

7 years agoDRILL-3778: Add missed part of DRILL-3160 (making JDBC Javadoc available).
dbarclay [Mon, 14 Sep 2015 17:54:34 +0000 (10:54 -0700)] 
DRILL-3778: Add missed part of DRILL-3160 (making JDBC Javadoc available).

This closes #158

Configured Javadoc generation (title, package groups, version in headers).
Added link to JDBC page in Drill documentation site.
Edited/fixed some JDBC Javadoc comments.
Added explicit SQLConversionOverflowException to throws clauses for Javadoc
Added some imports for Javadoc references.

Fixed a couple Javadoc syntax errors.
Fixed POM indentation.

7 years agoDRILL-3781: Group by system function in schema-based table.
Jinfeng Ni [Mon, 21 Sep 2015 21:59:03 +0000 (14:59 -0700)] 
DRILL-3781:  Group by system function in schema-based table.

Fix is in Calcite-886. Add unit test and bump forked Calcite version in Drill.

7 years agoDRILL-3811: AtomicRemainder incorrectly accounts for transferred allocations
adeneche [Sun, 20 Sep 2015 19:11:55 +0000 (12:11 -0700)] 
DRILL-3811: AtomicRemainder incorrectly accounts for transferred allocations

this closes #163