drill.git
6 years ago[maven-release-plugin] prepare release drill-1.7.0 1.7.0 drill-1.7.0
Aman Sinha [Thu, 23 Jun 2016 18:43:46 +0000 (11:43 -0700)] 
[maven-release-plugin] prepare release drill-1.7.0

6 years agoAdded Aman's GPG key.
Aman Sinha [Mon, 20 Jun 2016 21:39:04 +0000 (14:39 -0700)] 
Added Aman's GPG key.

6 years agoDRILL-4733: max(dir0) reading more columns than necessary
Arina Ielchiieva [Tue, 21 Jun 2016 12:33:32 +0000 (12:33 +0000)] 
DRILL-4733: max(dir0) reading more columns than necessary

6 years agoDRILL-2593: 500 error when crc for a query profile is out of sync
Arina Ielchiieva [Sun, 12 Jun 2016 13:48:54 +0000 (13:48 +0000)] 
DRILL-2593: 500 error when crc for a query profile is out of sync

6 years agoDRILL-4716: status.json doesn't work in drill ui
Arina Ielchiieva [Sun, 12 Jun 2016 16:10:52 +0000 (16:10 +0000)] 
DRILL-4716: status.json doesn't work in drill ui

6 years agoDRILL-4701: Fix log name and missing lines in logs on Web UI
Arina Ielchiieva [Wed, 1 Jun 2016 11:16:31 +0000 (11:16 +0000)] 
DRILL-4701: Fix log name and missing lines in logs on Web UI

6 years agoDRILL-3474: Add implicit file columns support
Arina Ielchiieva [Mon, 18 Apr 2016 16:36:52 +0000 (19:36 +0300)] 
DRILL-3474: Add implicit file columns support

6 years agoDRILL-2385: Count on complex objects failed with missing function implementation...
Vitalii Diravka [Thu, 16 Jun 2016 16:45:20 +0000 (16:45 +0000)] 
DRILL-2385: Count on complex objects failed with missing function implementation - added MapHolder, ListHolder; - added testCountComplexObjects() unit test.

6 years agoDRILL-4727: [Addendum] Exclude netty from HBase Client's transitive dependencies 528/head
Aditya Kishore [Fri, 17 Jun 2016 22:13:48 +0000 (15:13 -0700)] 
DRILL-4727: [Addendum] Exclude netty from HBase Client's transitive dependencies

Excluded `netty-all` from the list of transitive dependencies pulled by `mapr-hbase` in `mapr` profile.

6 years agoDRILL-4725: Improvements to InfoSchema RecordGenerator needed for DRILL-4714 524/head
vkorukanti [Thu, 9 Jun 2016 07:00:31 +0000 (00:00 -0700)] 
DRILL-4725: Improvements to InfoSchema RecordGenerator needed for DRILL-4714

1. Add support for pushing the filter on following fields into InfoSchemaRecordGenerator:
   - CATALOG_NAME
   - COLUMN_NAME

2. Pushdown LIKE with ESCAPE. Add test TestInfoSchemaFilterPushDown#testFilterPushdown_LikeWithEscape

3. Add a method visitCatalog() to InfoSchemaRecordGenerator to decide whether to explore the catalog or not

4. Refactor CATALOG_DESCRIPTION and CATALOG_CONNECT as constant strings in InfoSchemaConstants.java

5. Update TestInfoSchemaFilterPushDown#testPartialFilterPushDownWithProject as
   we are now pushing the filter on COLUMN_NAME field

6. Cleanup:
   Rename RecordGenerator -> InfoSchemaRecordGenerator
   Add comments in RecordGenerator
   Rename SelectedTable -> InfoSchemaTableType

this closes #524

Change-Id: I0b2e16d04cb72fe3ce5961f5f357a00655f1cb05

6 years agoDRILL-4727: Exclude netty from HBase Client's transitive dependencies 525/head
Aditya Kishore [Thu, 16 Jun 2016 20:39:21 +0000 (13:39 -0700)] 
DRILL-4727: Exclude netty from HBase Client's transitive dependencies

Excluded `netty-all` from the list of transitive dependencies pulled by `hbase-client`

Closes #525.

6 years agoDRILL-4573: Fixed issue with string functions when input contains non-ASCII characters.
jean-claude cote [Sat, 14 May 2016 22:02:57 +0000 (18:02 -0400)] 
DRILL-4573: Fixed issue with string functions when input contains non-ASCII characters.

Close apache/drill#512

6 years agoDRILL-4199: Add Support for HBase 1.X 443/head
Aditya Kishore [Sun, 12 Jun 2016 23:28:52 +0000 (16:28 -0700)] 
DRILL-4199: Add Support for HBase 1.X

Highlights of the changes:

* Replaced the old HBase APIs (HBaseAdmin/HTable) with the new HBase 1.1 APIs (Connection/Admin/Table).
* Added HBaseConnectionManager class which which manages the life-cycle of HBase connections inside a Drillbit process.
* Updated HBase dependencies version to 1.1.3 and 1.1.1-mapr-1602-m7-5.1.0 for default and "mapr" profiles respectively.
* Added `commons-logging` dependency in the `provided` scope to allow HBase test cluster to come up for Unit tests.
* Relaxed banned dependency rule for `commons-logging` library for `storage-hbase` module alone, in provided scope only.
* Removed the use of many deprecated APIs throughout the modules code.
* Added some missing test to HBase storage plugin's test suit.
* Move the GuavaPatcher code to main code execution path.
* Log a message if GuavaPatcher fails instead of exiting.

All unit tests are green.

Closes #443

6 years agoDRILL-4694: CTAS in JSON format produces extraneous NULL fields Changed behavior...
Parth Chandra [Thu, 2 Jun 2016 00:19:03 +0000 (17:19 -0700)] 
DRILL-4694: CTAS in JSON format produces extraneous NULL fields Changed behavior of JSON CTAS to skip fields if the value is null. Added an option "store.json.writer.skip_null_fields" to enable old behavior.

6 years agoDRILL-4607: - Fix unittest failure. Janino cannot compile a function that uses generi... 513/head
Parth Chandra [Fri, 3 Jun 2016 00:23:13 +0000 (17:23 -0700)] 
DRILL-4607: - Fix unittest failure. Janino cannot compile a function that uses generics; so replaced the implementation of StringFunctions.Split to not use any.

6 years agoDRILL-4607: Add a split function that allows to separate string by a delimiter
Alicia Alvarez [Fri, 15 Apr 2016 18:07:47 +0000 (11:07 -0700)] 
DRILL-4607: Add a split function that allows to separate string by a delimiter

This closes #506

6 years agoDRILL-4693: Ensure final column re-ordering is done if any select list expression...
Aman Sinha [Wed, 25 May 2016 22:33:56 +0000 (15:33 -0700)] 
DRILL-4693:  Ensure final column re-ordering is done if any select list expression is convert_fromjson.

close apache/drill#508

6 years agoDRILL-4143: parquet metadata is only opened in drillbituser (process user)'s security... 535/head
chunhui-shi [Thu, 7 Apr 2016 07:05:22 +0000 (00:05 -0700)] 
DRILL-4143: parquet metadata is only opened in drillbituser (process user)'s security context. Adding unit test for impersonated metadata update.

6 years agoDRILL-4679: When convert() functions are present, ensure that ProjectRecordBatch...
Aman Sinha [Tue, 17 May 2016 21:35:06 +0000 (14:35 -0700)] 
DRILL-4679: When convert() functions are present, ensure that ProjectRecordBatch produces a schema even for empty result set.

Add unit tests

Modify doAlloc() to accept record count parameter (addresses review comment)

6 years agoDRILL-4676: Foreman no longer uses a CountDownLatch and relies on the EventProcessor...
adeneche [Mon, 16 May 2016 20:19:18 +0000 (13:19 -0700)] 
DRILL-4676: Foreman no longer uses a CountDownLatch and relies on the EventProcessor instead

as part of this change Foreman.ResponseSendListener no longer calls Foreman.moveToState() as it doesn't make any difference
at this point.

this closes #503

6 years agoDRILL-4654: Add new metrics to the MetricRegistry
Sudheesh Katkam [Sat, 14 May 2016 01:16:15 +0000 (18:16 -0700)] 
DRILL-4654: Add new metrics to the MetricRegistry

+ New metrics:
  - drill.queries.enqueued
    number of queries that have been submitted to the drillbit but have
    not started
  - drill.queries.running
    number of running queries for which this drillbit is the foreman
  - drill.queries.completed
    number of completed queries (or cancelled or failed) for which this
    drillbit was the foreman
  - drill.fragments.running
    number of query fragments that are running in the drillbit
  - drill.allocator.root.used
    amount of memory used in bytes by the internal memory allocator
  - drill.allocator.root.peak
    peak amount of memory used in bytes by the internal memory allocator
  - fd.usage
    ratio of used to total file descriptors (on *nix systems)
+ Rename "current" to "used" for RPC allocator current memory usage to
  follow convention
+ Borrow SystemPropertyUtil class from Netty
+ Configure DrillMetrics through system properties
+ Remove unused methods and imports

closes #495

6 years agoDRILL-4657: Rank() will return wrong results if a frame of data is too big (more...
adeneche [Wed, 4 May 2016 19:08:36 +0000 (12:08 -0700)] 
DRILL-4657: Rank() will return wrong results if a frame of data is too big (more than 2 batches)

this closes #499

6 years agoDRILL-4132 Ability to submit simple type of physical plan directly to EndPoint DrillB...
Yuliya Feldman [Thu, 4 Feb 2016 22:09:21 +0000 (14:09 -0800)] 
DRILL-4132 Ability to submit simple type of physical plan directly to EndPoint DrillBit for execution. There are multiple changes to achieve this: 1. During physical planning split single plan into multiple based on the number of minor fragments of the Leaf Major fragment. a. Removing exchange operators during planning b. Producing just root fragments (that will be also leaf fragments) 2. Each fragment can be executed against Drillbit it is assigned to, so to keep locality Design document can be found in the JIRA: DRILL-4132

6 years agoDRILL-4652: Drill native client - fix compile issue when the latest git commit descri...
Patrick Wong [Tue, 3 May 2016 22:19:26 +0000 (15:19 -0700)] 
DRILL-4652: Drill native client - fix compile issue when the latest git commit description has a period or quotation mark in it

This closes #494

6 years agoDRILL-4584: JDBC/ODBC Client IP in Drill audit logs - the format of added field in...
Vitalii Diravka [Fri, 8 Apr 2016 08:01:37 +0000 (11:01 +0300)] 
DRILL-4584: JDBC/ODBC Client IP in Drill audit logs - the format of added field in log files is "remoteAddress":"192.168.121.1:58984"

This closes #475

6 years agoDRILL-3894: Upgrade functions MaxDir, MinDir... Optional filename parameter - added...
Vitalii Diravka [Tue, 5 Apr 2016 15:07:29 +0000 (15:07 +0000)] 
DRILL-3894: Upgrade functions MaxDir, MinDir... Optional filename parameter - added implementations of functions MAXDIR, IMAXDIR, MINDIR, IMINDIR with one (schema) without filename argument; - added UTest for results comparison of using Query Directory Functions with one and two arguments.

This closes #467

6 years agoDRILL-2100: Added deleting temporary spill directories when query is finished.
Vitalii Diravka [Mon, 28 Mar 2016 18:05:22 +0000 (18:05 +0000)] 
DRILL-2100: Added deleting temporary spill directories when query is finished.

This closes #454

6 years agoDRILL-4571: Add link to local Drill logs from the web UI
Arina Ielchiieva [Thu, 31 Mar 2016 15:43:25 +0000 (18:43 +0300)] 
DRILL-4571: Add link to local Drill logs from the web UI

This closes #472

6 years agoUpdated instructions for Linux build of the Drill Native Client
Patrick Wong [Tue, 3 May 2016 01:30:34 +0000 (18:30 -0700)] 
Updated instructions for Linux build of the Drill Native Client

1. CMake 3.0 is required, rather than CMake 2.8
2. Added some info that is helpful for the case where you build Boost the hard way.

This closes #492

6 years agoDRILL-4577: Construct a specific path for querying all the tables from a hive database 461/head
Hsuan-Yi Chu [Mon, 4 Apr 2016 23:05:04 +0000 (16:05 -0700)] 
DRILL-4577: Construct a specific path for querying all the tables from a hive database

6 years agoDRILL-4529: Force $SUM0 to be used when Window Sum is supposed to returned non-nullab... 447/head
Hsuan-Yi Chu [Sun, 27 Mar 2016 23:18:21 +0000 (16:18 -0700)] 
DRILL-4529: Force $SUM0 to be used when Window Sum is supposed to returned non-nullable type

6 years agoDRILL-4573: Zero copy LIKE, REGEXP_MATCHES, SUBSTR
jean-claude cote [Sat, 2 Apr 2016 03:37:00 +0000 (23:37 -0400)] 
DRILL-4573: Zero copy LIKE, REGEXP_MATCHES, SUBSTR

6 years agoDRILL-3522: Fix for sporadic Mongo errors
Adam [Tue, 21 Jul 2015 02:44:21 +0000 (12:44 +1000)] 
DRILL-3522: Fix for sporadic Mongo errors

6 years agoDRILL-4437: Operator unit test framework
Jason Altekruse [Fri, 26 Feb 2016 22:55:30 +0000 (14:55 -0800)] 
DRILL-4437: Operator unit test framework

Closes #394

6 years agoDRILL-4442: Move getSV2 and getSV4 methods to VectorAccessible
Jason Altekruse [Fri, 26 Feb 2016 22:11:47 +0000 (14:11 -0800)] 
DRILL-4442: Move getSV2 and getSV4 methods to VectorAccessible

Up one level from previous location RecordBatch, most implementations
already implement the method as they implement RecordBatch rather than
VectorAccessible itself. Add unsupported operation exception to others.

6 years agoDRILL-4448: Clean up deserialization of oderings in sorts
Jason Altekruse [Thu, 25 Feb 2016 03:37:21 +0000 (19:37 -0800)] 
DRILL-4448: Clean up deserialization of oderings in sorts

Fix sort operator deserialization and validation to respect existing
contract specified in the tests.

6 years agoDRILL-4445: Standardize the Physical and Logical plan nodes to use Lists instead...
Jason Altekruse [Thu, 11 Feb 2016 01:36:47 +0000 (17:36 -0800)] 
DRILL-4445: Standardize the Physical and Logical plan nodes to use Lists instead of arrays for their inputs

Remove some extra translation logic used to move between the
two representations.

TODO - look back the the Join logical node, has two JsonCreator annotations,
but only one will be used. Not sure if the behavior of which is chosen
is considered documented behavior, should just fix it on our end.

6 years agoDRILL-4237, DRILL-4478: Implement hash to use murmur3 and add correspondent unit...
chunhui-shi [Tue, 15 Mar 2016 17:36:54 +0000 (10:36 -0700)] 
DRILL-4237, DRILL-4478: Implement hash to use murmur3 and add correspondent unit tests

+ Avoid object or extra buffer creation
+ Clean up tests

closes #485

6 years agoDRILL-4459: Resolve SchemaChangeException while querying hive json table
Vitalii Diravka [Thu, 10 Mar 2016 14:52:28 +0000 (14:52 +0000)] 
DRILL-4459: Resolve SchemaChangeException while querying hive json table

- Replace drill var16char to varchar datatype for hive string datatype
- Change testGenericUDF() and testUDF() to use VarChar instead of Var16Char
- Add unit test for hive GET_JSON_OBJECT UDF

closes #431

6 years agoDRILL-4390: Uses Resource where Drill favicon is located for static assets
Laurent Goujon [Wed, 17 Feb 2016 00:47:17 +0000 (16:47 -0800)] 
DRILL-4390: Uses Resource where Drill favicon is located for static assets

Drill Webserver uses the first jar containing a rest/static directory to
find its static assets. In case of another jar containing this directory, it
might cause the webserver to return 404 errors.

This configures the server to use the resource containing the Drill favicon
as the place to look for all static resources.

this closes #378

6 years agoDRILL-3714: Avoid cascading disconnection when a single connection is broken. 430/head
Jacques Nadeau [Tue, 5 Apr 2016 23:36:25 +0000 (16:36 -0700)] 
DRILL-3714: Avoid cascading disconnection when a single connection is broken.

- Move the coordination id management to be connection level instead of RpcBus level
- Rename CoordinationQueue to a more appropriate name: RequestIdMap
- Simplify locking and memory overhead of RequestIdMap. It used to be that this would accessed by a large number of threads concurrently. We modified the behavior so that it is only accessed by two threads at most. Rather than have memory overhead of ConcurrentHashMap, switch to simple locking approach since contention should be low.
- Update all methods associated with coordination to improve names as well as add javadocs. Move these methods to the RemoteConnection.
- Consolidate the two different close handlers into a single, ordered close handler managed inside the connection.
- Add better javadoc around the close method of RemoteConnection
- Add some preconditions checks.
- Update the HPPC version in the base memory module since it conflicts with the one in the java-exec module.

This closes #463.

6 years agoDRILL-4593: Remove OldAssignmentCreator in FileSystemPlugin
vkorukanti [Thu, 7 Apr 2016 21:23:07 +0000 (14:23 -0700)] 
DRILL-4593: Remove OldAssignmentCreator in FileSystemPlugin

+ Remove dead code in ParquetGroupScan

this closes #473

6 years agoDRILL-4446: Support mandatory work assignment to endpoint requirements of operators
vkorukanti [Wed, 2 Mar 2016 21:08:36 +0000 (13:08 -0800)] 
DRILL-4446: Support mandatory work assignment to endpoint requirements of operators

6 years agoDRILL-4592: Explain plan statement should show plan in WebUI 468/head
Jinfeng Ni [Thu, 7 Apr 2016 18:24:04 +0000 (11:24 -0700)] 
DRILL-4592: Explain plan statement should show plan in WebUI

6 years agoDRILL-4589: Reduce planning time for file system partition pruning by reducing filter...
Jinfeng Ni [Thu, 25 Feb 2016 18:13:43 +0000 (10:13 -0800)] 
DRILL-4589: Reduce planning time for file system partition pruning by reducing filter evaluation overhead

6 years agoDRILL-4523: Disallow using loopback address in distributed mode
Arina Ielchiieva [Thu, 24 Mar 2016 16:46:06 +0000 (16:46 +0000)] 
DRILL-4523: Disallow using loopback address in distributed mode

closes #445

6 years agoDRILL-4544: Improve error messages for REFRESH TABLE METADATA command
Arina Ielchiieva [Mon, 28 Mar 2016 10:55:56 +0000 (10:55 +0000)] 
DRILL-4544: Improve error messages for REFRESH TABLE METADATA command
1. Added error message when storage plugin or workspace does not exist
2. Updated error message when refresh metadata is not supported
3. Unit tests

closes #448

6 years agoDRILL-4588: Enable JMX reporting
Sudheesh Katkam [Wed, 6 Apr 2016 23:41:52 +0000 (16:41 -0700)] 
DRILL-4588: Enable JMX reporting

closes #469

6 years agoDRILL-3743: Fail active result listeners if server connection is closed
Sudheesh Katkam [Tue, 5 Apr 2016 23:04:44 +0000 (16:04 -0700)] 
DRILL-3743: Fail active result listeners if server connection is closed

+ Remove dead code
+ Improve error and logging messages

closes #460

6 years agoDRILL-4551: Implement new functions (cot, regex_matches, split_part, isdate) 452/head
Jason Altekruse [Mon, 28 Mar 2016 18:55:11 +0000 (11:55 -0700)] 
DRILL-4551: Implement new functions (cot, regex_matches, split_part, isdate)

6 years agoDRILL-4549: Add support for more truncation units in date_trunc function 450/head
vkorukanti [Mon, 28 Mar 2016 18:09:34 +0000 (11:09 -0700)] 
DRILL-4549: Add support for more truncation units in date_trunc function

6 years agoDRILL-4531: Add a Drill customized rule for pushing filter past aggregate 444/head
Jinfeng Ni [Wed, 23 Mar 2016 00:57:02 +0000 (17:57 -0700)] 
DRILL-4531: Add a Drill customized rule for pushing filter past aggregate

6 years agoDRILL-4317: Exceptions on SELECT and CTAS with large CSV files
adeneche [Wed, 16 Mar 2016 13:47:18 +0000 (14:47 +0100)] 
DRILL-4317: Exceptions on SELECT and CTAS with large CSV files

this closes #432

6 years agoDRILL-3623: For limit 0 queries, optionally use a shorter execution path when result...
Sudheesh Katkam [Tue, 22 Mar 2016 22:21:51 +0000 (15:21 -0700)] 
DRILL-3623: For limit 0 queries, optionally use a shorter execution path when result column types are known

+ "planner.enable_limit0_optimization" option is disabled by default

+ Print plan in PlanTestBase if TEST_QUERY_PRINTING_SILENT is set
+ Fix DrillTestWrapper to verify expected and actual schema
+ Correct the schema of results in TestInbuiltHiveUDFs#testXpath_Double

This closes #405

6 years agoBump calcite version to 1.4.0-drill-r11 397/head
Hsuan-Yi Chu [Fri, 18 Mar 2016 04:54:05 +0000 (21:54 -0700)] 
Bump calcite version to 1.4.0-drill-r11

6 years agoDRILL-4372: (continued) Support for Window functions:
Hsuan-Yi Chu [Mon, 14 Mar 2016 23:11:10 +0000 (16:11 -0700)] 
DRILL-4372: (continued) Support for Window functions:
- CUME_DIST
- DENSE_RANK
- PERCENT_RANK
- RANK
- ROW_NUMBER
- NTILE
- LEAD
- LAG
- FIRST_VALUE
- LAST_VALUE

6 years agoDRILL-4372: (continued) Add option to disable/enable function output type inference
Hsuan-Yi Chu [Wed, 9 Mar 2016 01:57:36 +0000 (17:57 -0800)] 
DRILL-4372: (continued) Add option to disable/enable function output type inference

6 years agoDRILL-4372: (continued) Type inference for HiveUDFs
Hsuan-Yi Chu [Fri, 4 Mar 2016 04:14:59 +0000 (20:14 -0800)] 
DRILL-4372: (continued) Type inference for HiveUDFs

6 years agoDRILL-4372: Expose the functions return type to Drill
Hsuan-Yi Chu [Fri, 4 Mar 2016 06:38:04 +0000 (22:38 -0800)] 
DRILL-4372: Expose the functions return type to Drill

- Drill-Calite version update:
This commit needs to have Calcite's patch (CALCITE-1062) to plugin customized SqlOperator.

- FunctionTemplate
Add FunctionArgumentNumber annotation. This annotation element tells if the number of argument(s) is fixed or arbitrary (e.g., String concatenation function).

Due to this modification, there are some minor changes in DrillFuncHolder, DrillFunctionRegistry and FunctionAttributes.

- Checker
Add a new Checker (which Calcite uses to validate the legitimacy of the number of argument(s) for a function) to allow functions with arbitrary arguments to pass Caclite's validation

- Type conversion between Drill and Calcite
DrillConstExector is given a static method getDrillTypeFromCalcite() to convert Calcite types to Drill's.

- Extract function's return type inference
Unlike other functions, Extract function's return type can be determined solely based on the first argument. A logic is added in to allow this inference to happen

- DrillCalcite wrapper:
From the aspects of return type inference and argument type checks, Calcite's mechanism is very different from Drill's. In addition, currently, there is no straightforward way for Drill to plug-in customized mechanisms to Calcite. Thus, wrappers are provided to serve the objective.

Except for the mechanisms of type inference and argument type checks, these wrappers just forward any method calls to the wrapped SqlOpertor, SqlFuncion or SqlAggFunction to respond.

A interface DrillCalciteSqlWrapper is also added for the callers of the three wrappers to get the wrapped objects easier.

Due to these wrappers, UnsupportedOperatorsVisitor is modified in a minor manner.

- Calcite's SqlOpertor, SqlFuncion or SqlAggFunction are wrapped in DrillOperatorTable
Instead of returning Caclite's native SqlOpertor, SqlFuncion or SqlAggFunction, return the wrapped ones to ensure customized behaviors can be adopted.

- Type inference mechanism
This mechanism is used across all SqlOpertor, SqlFuncion or SqlAggFunction. Thus, it is factored out as its own method in TypeInferenceUtils

- Upgrade Drill-Calcite

Bump version number to 1.4.0-drill-test-r16

- Implement two argument version of lpad, rpad

- Implement one argument version of ltrim, rtrim, btrim

6 years agoDRILL-4501: Complete MapOrListWriter for all supported data types
Aditya Kishore [Sat, 12 Mar 2016 19:12:34 +0000 (11:12 -0800)] 
DRILL-4501: Complete MapOrListWriter for all supported data types

Closes #427

6 years agoDRILL-4484: NPE when querying empty directory
adeneche [Thu, 10 Mar 2016 09:40:06 +0000 (10:40 +0100)] 
DRILL-4484: NPE when querying  empty directory

6 years agoDRILL-4376: Wrong results when doing a count(*) on part of directories with metadata...
adeneche [Wed, 9 Mar 2016 12:44:02 +0000 (13:44 +0100)] 
DRILL-4376: Wrong results when doing a count(*) on part of directories with metadata cache

6 years agoDRILL-4050: Add zip archives to the list of artifacts in verify_release.sh
Aditya Kishore [Mon, 14 Mar 2016 22:15:38 +0000 (15:15 -0700)] 
DRILL-4050: Add zip archives to the list of artifacts in verify_release.sh

This enhanced version of the script allows integrated download and verification of a Drill release. It can be used to verify both the main release artifacts and maven repository artifacts.

For example, to verify the 1.6 rc0 release artifacts, I ran

./verify_release.sh https://repository.apache.org/content/repositories/orgapachedrill-1030/ /tmp/drill-1.6/maven/
./verify_release.sh http://home.apache.org/~parthc/drill/releases/1.6.0/rc0/ /tmp/drill-1.6/main/

If I had pre-downloaded the files in the respective folders, I'd run

./verify_release.sh /tmp/drill-1.6/maven/
./verify_release.sh /tmp/drill-1.6/main/

Finally, run with `-nv` option to reduce the verbosity of the output.

Closes #249.

6 years agoDRILL-3745: Hive CHAR not supported
Arina Ielchiieva [Fri, 19 Feb 2016 17:03:52 +0000 (17:03 +0000)] 
DRILL-3745: Hive CHAR not supported

6 years agoDRILL-4479: For empty fields under all_text_mode enabled (a) use varchar for the...
Aman Sinha [Tue, 8 Mar 2016 17:27:32 +0000 (09:27 -0800)] 
DRILL-4479: For empty fields under all_text_mode enabled (a) use varchar for the default columns and (b) ensure we create fields corresponding to all columns.

close apache/drill#420

6 years agoDRILL-4490: Ensure the count generated by ConvertCountToDirectScan is non-nullable
Hsuan-Yi Chu [Thu, 10 Mar 2016 01:25:11 +0000 (17:25 -0800)] 
DRILL-4490: Ensure the count generated by ConvertCountToDirectScan is non-nullable

6 years agoAdded Parth's GPG Key
Parth Chandra [Sun, 13 Mar 2016 16:50:54 +0000 (09:50 -0700)] 
Added Parth's GPG Key

6 years agoDRILL-4476: Allow UnionAllRecordBatch to manager situations where left input side...
Hsuan-Yi Chu [Fri, 4 Mar 2016 21:50:02 +0000 (13:50 -0800)] 
DRILL-4476: Allow UnionAllRecordBatch to manager situations where left input side or both sides come(s) from empty source(s).

close apache/drill#407

6 years agoUpdate version to 1.7.0-SNAPSHOT
Parth Chandra [Fri, 11 Mar 2016 01:02:16 +0000 (17:02 -0800)] 
Update version to 1.7.0-SNAPSHOT

6 years agoDRILL-4482: Fix Avro nested field selection regression
Jason Altekruse [Tue, 8 Mar 2016 22:16:03 +0000 (14:16 -0800)] 
DRILL-4482: Fix Avro nested field selection regression

Update some of the Avro tests to properly verify their results,
others still need to be fixed. These will be addressed in DRILL-4110.

Closes #419

6 years agoDRILL-4441: Fix varchar data read out of Avro filtering incorrectly due to metadata bug
Jason Altekruse [Fri, 26 Feb 2016 17:55:05 +0000 (09:55 -0800)] 
DRILL-4441: Fix varchar data read out of Avro filtering incorrectly due to metadata bug

The precision of the Varchar datatype was not being set causing inconsistent
truncation of values to the default length of 1. Fixed the same issue with varbinary.

The test framework was previously taking a string as the baseline for a binary value,
which cannot express all possible values. Fixed the test to intstead use a byte array.
Thie required updating the hive tests that were using the old method of specifying
baselines with a String.

Fix cast to varbinary when reading from a data source with schema needed for writing
a test.

Updated patch to remove varchar lengths from table creation.
This issue was fixed more generally by DRILL-4465, which provides a default
type length for varchar and varbinary during the setup of calcite. This update now
just provides tests to verify the fix in this case.

Closes #393

6 years agoDRILL-4493 - Fixed issues in various POMs with MapR profile
Aditya Kishore [Wed, 9 Mar 2016 08:38:33 +0000 (00:38 -0800)] 
DRILL-4493 - Fixed issues in various POMs with MapR profile
This closes #421

6 years agoDRILL-4487: add unit test for DRILL-4449
adeneche [Tue, 8 Mar 2016 15:54:31 +0000 (16:54 +0100)] 
DRILL-4487: add unit test for DRILL-4449

6 years agoDRILL-4474: Ensure that ConvertCountToDirectScan does not push through project when...
Jinfeng Ni [Tue, 8 Mar 2016 22:15:27 +0000 (14:15 -0800)] 
DRILL-4474: Ensure that ConvertCountToDirectScan does not push through project when nullable input of count is not RexInputRef
This closes #416

6 years agoDRILL-4474: Ensure that ConvertCountToDirectScan only pushes through project when...
Jacques Nadeau [Fri, 4 Mar 2016 21:27:26 +0000 (13:27 -0800)] 
DRILL-4474: Ensure that ConvertCountToDirectScan only pushes through project when project is trivial.
This closes #406

6 years agoDRILL-4485 - MapR profile - switch to MapR 5.1.0, and improve compatibility with...
Patrick Wong [Tue, 8 Mar 2016 02:22:08 +0000 (18:22 -0800)] 
DRILL-4485 - MapR profile - switch to MapR 5.1.0, and improve compatibility with maprfs storage format and MapR DB storage plugin.
 This closes #417

6 years agoDRILL-4443: MIN/MAX on VARCHAR throw a NullPointerException
adeneche [Fri, 26 Feb 2016 21:41:36 +0000 (13:41 -0800)] 
DRILL-4443: MIN/MAX on VARCHAR throw a NullPointerException

6 years agoDRILL-4332: Makes vector comparison order stable in test framework
Laurent Goujon [Tue, 23 Feb 2016 22:14:37 +0000 (14:14 -0800)] 
DRILL-4332: Makes vector comparison order stable in test framework

In the test framework, a vector is a map of <String, Object>. When comparing
actual values with baseline, the comparison is made column by column, but
a HashMap key ordering is not guaranteed, and the ordering actually changed
between Java7 and Java8 in Oracle/OpenJDK.

Replacing HashMap with TreeMap which has a guaranteed ordering by design.

Small update by jason during merge, fixed test failure on JDK 7 due to map key ordering,
just replaced two more uses of HashMap with TreeMap.

Closes #389

6 years agoDRILL-4486: Fix expression serialization escaping
Steven Phillips [Sat, 5 Mar 2016 04:29:26 +0000 (20:29 -0800)] 
DRILL-4486: Fix expression serialization escaping

Closes #412

6 years agoAdding Jason's GPG key
Jason Altekruse [Mon, 1 Feb 2016 02:35:42 +0000 (18:35 -0800)] 
Adding Jason's GPG key

6 years agoDRILL-4375: Fix the maven release profile
Jason Altekruse [Tue, 2 Feb 2016 07:17:37 +0000 (23:17 -0800)] 
DRILL-4375: Fix the maven release profile

This generated pom file was being discovered and maven was trying to
run the target directory in jdbc-all as a submodule.

This change reverts to the default output location (the module root)
and adds corresponding .gitignore and RAT exclude entries. NOTE:
this is considered bad practice as generated files should appear in
the target directory and be removed upon a maven clean. This default
location is considered to be a known shortcoming of the shade plugin.

Also updated integration test for the jdbc-all jar with small
path change, as changing the location of the dependency-reduced-pom.xml
actually changed the directory the test was being executed from.

Closes #402

6 years agoDRILL-4483: Fix text plan regression in query profiles 411/head
vkorukanti [Mon, 7 Mar 2016 23:08:58 +0000 (15:08 -0800)] 
DRILL-4483: Fix text plan regression in query profiles

6 years agoDRILL-4313: C++ Client - Thread safe Logging. Improved Drill bit selection.
Parth Chandra [Fri, 12 Feb 2016 23:42:53 +0000 (15:42 -0800)] 
DRILL-4313: C++ Client - Thread safe Logging.  Improved Drill bit selection.
 - Update random drill bit selection. Shuffle the list initially, then round robin. Add Utility methods to get random numbers and to shuffle and add vectors. Whitespace cleanup
 - Add Git properties to build and print to log.
 - Add interface to get error based on query handle.
 - Add support for Pooled connections. Allows switching between pooled and unpooled connections based on environment variables

6 years agoDRILL-4457: Difference in results returned by window function over BIGINT data
adeneche [Mon, 7 Mar 2016 14:08:08 +0000 (15:08 +0100)] 
DRILL-4457: Difference in results returned by window function over BIGINT data

this closes #410

6 years agoDRILL-4281: Support authorized proxy users to impersonate other users 400/head
Sudheesh Katkam [Sat, 5 Mar 2016 23:18:28 +0000 (15:18 -0800)] 
DRILL-4281: Support authorized proxy users to impersonate other users

closes #400

6 years agoDRILL-4449: Wrong results when metadata cache is used with specific set of queries
adeneche [Thu, 25 Feb 2016 03:19:09 +0000 (19:19 -0800)] 
DRILL-4449: Wrong results when metadata cache is used with specific set of queries

close apache/drill#392

6 years agoDRILL-4467: Fix field ordering issue in PrelUtil
Jacques Nadeau [Fri, 4 Mar 2016 11:04:40 +0000 (03:04 -0800)] 
DRILL-4467: Fix field ordering issue in PrelUtil

6 years agoDRILL-4465: Simplify Calcite parsing & planning integration 401/head
Jacques Nadeau [Thu, 25 Feb 2016 18:45:40 +0000 (10:45 -0800)] 
DRILL-4465: Simplify Calcite parsing & planning integration

- Canonicalize Planning phases with PlannerPhase enumeration
- Canonicalize PlannerType transforms
- Remove dependency on Calcite's Frameworks.Planner since Drill need stop heavily customize interactions
- Update AbstractStoragePlugin to implement a phase-aware planning rule injection behavior.
- Avoid (or at least reduce) duplicated registerSchemas() invocations

This closes #401.

6 years agoDRILL-4354: Remove sessions in anonymous (auth disabled) WebUI access
vkorukanti [Tue, 2 Feb 2016 14:48:18 +0000 (06:48 -0800)] 
DRILL-4354: Remove sessions in anonymous (auth disabled) WebUI access

6 years agoDRILL-4410: ListVector should initialize bits in allocateNew
Minji Kim [Thu, 18 Feb 2016 06:29:11 +0000 (22:29 -0800)] 
DRILL-4410: ListVector should initialize bits in allocateNew

Remove large arrays.json files and generate it in /tmp, Check test results
Add another unit test for ListVector in TestValueVector, and use tempDir in TestComplexTypeReader

Closes #380

6 years agoDRILL-4384: Fix Profile regressions
Jacques Nadeau [Sun, 21 Feb 2016 18:29:09 +0000 (10:29 -0800)] 
DRILL-4384: Fix Profile regressions

6 years agoDRILL-4383: Allow custom configurations to be specified for a FileSystem plugin
Jason Altekruse [Fri, 12 Feb 2016 00:52:38 +0000 (16:52 -0800)] 
DRILL-4383: Allow custom configurations to be specified for a FileSystem plugin

add an example s3 plugin, disabled by default

Closes #375

6 years agoDRILL-4327: Fix rawtypes warnings in drill codebase
Laurent Goujon [Thu, 28 Jan 2016 03:01:13 +0000 (19:01 -0800)] 
DRILL-4327: Fix rawtypes warnings in drill codebase

Fixing most rawtypes warning issues in drill modules.

Closes #347

6 years agoDRILL-3488: Allow Java 1.8
adeneche [Tue, 23 Feb 2016 17:02:38 +0000 (09:02 -0800)] 
DRILL-3488: Allow Java 1.8

6 years agoDRILL-3688: Drill should honor "skip.header.line.count" and "skip.footer.line.count...
Arina Ielchiieva [Thu, 11 Feb 2016 17:16:30 +0000 (17:16 +0000)] 
DRILL-3688: Drill should honor "skip.header.line.count" and "skip.footer.line.count" attribute of Hive table

1. Functionality to skip header and footer lines while reading Hive data.
2. Unit tests.

6 years agoDRILL-4346: Handle NumberFormatException when casting empty string to int in hbase...
vataga [Tue, 2 Feb 2016 11:18:49 +0000 (13:18 +0200)] 
DRILL-4346: Handle NumberFormatException when casting empty string to int in hbase/maprdb

- Made replacing of functions casting for nullable input varbinary and var16char types are working (in CastFunctions.java setting 'drill.exec.functions.cast_empty_string_to_null = true' didn't work);
- Added new generated classes for casting varbinary and var16char to int (CastEmptyStringVarTypesToNullableNumeric template and Casts.tdd data);
- Created a test (in TestHBaseQueries.java) for checking an empty string to integer casting in maprdb/hbase table;
- Small design changes according to comments in review (import statements, prefixes ...).

This closes #384

6 years agoDRILL-3944: Drill MAXDIR Unknown variable or type FILE_SEPARATOR. This
Arina Ielchiieva [Tue, 2 Feb 2016 16:48:29 +0000 (18:48 +0200)] 
DRILL-3944: Drill MAXDIR Unknown variable or type FILE_SEPARATOR. This
closes #391

6 years agoDRILL-4434: Deprecate GroupScan.enforceWidth API 390/head
vkorukanti [Wed, 24 Feb 2016 22:13:21 +0000 (14:13 -0800)] 
DRILL-4434: Deprecate GroupScan.enforceWidth API

6 years agoDRILL-4423: TestCsvHeader#testEmptyFinalColumn is causing the unit test framework...
adeneche [Mon, 22 Feb 2016 20:48:11 +0000 (12:48 -0800)] 
DRILL-4423: TestCsvHeader#testEmptyFinalColumn is causing the unit test framework to print 10K rows in Stdout

this closes #387

6 years agoDRILL-4387: GroupScan or ScanBatchCreator should not use star column in case of skipA...
Jinfeng Ni [Fri, 12 Feb 2016 22:18:59 +0000 (14:18 -0800)] 
DRILL-4387: GroupScan or ScanBatchCreator should not use star column in case of skipAll query.

The skipAll query should be handled in RecordReader.