hive.git
3 months agoUpdate README.md branch-4.0.0-alpha-1 rel/release-4.0.0-alpha-1
Peter Vary [Tue, 22 Mar 2022 10:02:22 +0000 (11:02 +0100)] 
Update README.md

3 months agoUpdating release notes
Peter Vary [Tue, 22 Mar 2022 09:03:06 +0000 (10:03 +0100)] 
Updating release notes

3 months agoPreparing for 4.0.0-alpha-1 release
Peter Vary [Tue, 22 Mar 2022 08:48:08 +0000 (09:48 +0100)] 
Preparing for 4.0.0-alpha-1 release

3 months agoHIVE-26033: Repl Load fails with Wrong FS error. (#3100). (Ayush Saxena, reviewed...
Ayush Saxena [Tue, 22 Mar 2022 05:39:01 +0000 (11:09 +0530)] 
HIVE-26033: Repl Load fails with Wrong FS error. (#3100). (Ayush Saxena, reviewed by Mahesh Kumar Behera)

3 months agoHIVE-26049: Inconsistent TBL_NAME lengths in HMS schema (Janos Kovacs reviewed by...
Janos Kovacs [Mon, 21 Mar 2022 14:54:47 +0000 (15:54 +0100)] 
HIVE-26049: Inconsistent TBL_NAME lengths in HMS schema (Janos Kovacs reviewed by Peter Vary) (#3119)

3 months agoHIVE-26048: Missing quotation mark in findReadyToClean query (Antal Sinkovits, review...
Antal Sinkovits [Mon, 21 Mar 2022 12:20:41 +0000 (13:20 +0100)] 
HIVE-26048: Missing quotation mark in findReadyToClean query (Antal Sinkovits, reviewed by Karen Coppage and Denys Kuzmenko)

Closes #3118

3 months agoHIVE-26016: Remove duplicate table exists check in create_table_core api of HMSHandle...
Wechar Yu [Mon, 21 Mar 2022 10:18:48 +0000 (18:18 +0800)] 
HIVE-26016: Remove duplicate table exists check in create_table_core api of HMSHandler (Wechar Yu, reviewed by Rajesh Balamohan and Peter Vary) (#3085)

3 months agoHIVE-26042: Fix flaky streaming tests (Peter Vary, reviewed by Marton Bod) (#3114)
pvary [Mon, 21 Mar 2022 08:36:44 +0000 (09:36 +0100)] 
HIVE-26042: Fix flaky streaming tests (Peter Vary, reviewed by Marton Bod) (#3114)

3 months agoHIVE-25956: Non blocking RENAME TABLE implementation (Denys Kuzmenko, reviewed by...
Denys Kuzmenko [Mon, 21 Mar 2022 07:50:40 +0000 (08:50 +0100)] 
HIVE-25956: Non blocking RENAME TABLE implementation (Denys Kuzmenko, reviewed by Laszlo Pinter)

Closes #3022

3 months agoHIVE-25814: Add entry in replication_metrics table for skipped replication. (#2907...
Haymant Mangla [Mon, 21 Mar 2022 06:53:28 +0000 (12:23 +0530)] 
HIVE-25814: Add entry in replication_metrics table for skipped replication. (#2907). (Haymant Mangla, reviewed by Ayush Saxena)

3 months agoHIVE-26029: Upgrade netty to 4.1.75.Final due to CVE (Yu-Wen Lai via Naveen Gangam)
Yu-Wen Lai [Fri, 11 Mar 2022 19:48:00 +0000 (11:48 -0800)] 
HIVE-26029: Upgrade netty to 4.1.75.Final due to CVE (Yu-Wen Lai via Naveen Gangam)

3 months agoHIVE-26028: Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE (Yu-Wen Lai via Naveen...
Yu-Wen Lai [Fri, 11 Mar 2022 18:46:52 +0000 (10:46 -0800)] 
HIVE-26028: Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE (Yu-Wen Lai via Naveen Gangam)

3 months agoHIVE-26002: Preparing for 4.0.0-alpha-1 development (Peter Vary reviewed by Stamatis...
pvary [Thu, 17 Mar 2022 16:25:12 +0000 (17:25 +0100)] 
HIVE-26002: Preparing for 4.0.0-alpha-1 development (Peter Vary reviewed by Stamatis Zampetakis) (#3081)

3 months agoHIVE-26040: Fix DirectSqlUpdateStat.getNextCSIdForMPartitionColumnStatistics for...
pvary [Thu, 17 Mar 2022 11:25:13 +0000 (12:25 +0100)] 
HIVE-26040: Fix DirectSqlUpdateStat.getNextCSIdForMPartitionColumnStatistics for mssql (Peter Vary reviewed by Marton Bod) (#3112)

3 months agoHIVE-26025: Remove IMetaStoreClient#listPartitionNames which is not used (Zhihua...
dengzh [Wed, 16 Mar 2022 18:38:57 +0000 (02:38 +0800)] 
HIVE-26025: Remove IMetaStoreClient#listPartitionNames which is not used (Zhihua Deng reviewed by Peter Vary) (#3093)

3 months agoHIVE-25994: Analyze table runs into ClassNotFoundException-s (Alessandro Solimando...
Alessandro Solimando [Wed, 16 Mar 2022 13:32:02 +0000 (14:32 +0100)] 
HIVE-25994: Analyze table runs into ClassNotFoundException-s (Alessandro Solimando reviewed by Laszlo Bodor, Peter Vary) (#3095)

3 months agoDisable flaky test
Peter Vary [Wed, 16 Mar 2022 08:16:36 +0000 (09:16 +0100)] 
Disable flaky test

3 months agoRevert "HIVE-25845: Support ColumnIndexes for Parq files (#3094)"
Peter Vary [Mon, 14 Mar 2022 20:48:58 +0000 (21:48 +0100)] 
Revert "HIVE-25845: Support ColumnIndexes for Parq files (#3094)"

This reverts commit 88054ce553604a6d939149faf4d1c3b037706aba.

3 months agoHIVE-25963: Addendum: ported unit test fixes (Sourabh Goyal via Naveen Gangam)
Naveen Gangam [Tue, 15 Mar 2022 22:36:48 +0000 (18:36 -0400)] 
HIVE-25963: Addendum: ported unit test fixes (Sourabh Goyal via Naveen Gangam)

    Ported back commits from  https://github.com/apache/hive/pull/3040/commits
    that are missing in master branch

3 months agoHIVE-25146: JMH tests for Multi HT and parallel load (#3044) (Panagiotis Garefalakis...
Panagiotis Garefalakis [Mon, 14 Mar 2022 18:11:53 +0000 (11:11 -0700)] 
HIVE-25146: JMH tests for Multi HT and parallel load (#3044) (Panagiotis Garefalakis, reviewed by Ramesh Kumar)

* JMH tests for parallel HT load, configuration parameters include LOAD_THREADS_NUM, ROWS_NUM and JOIN_TYPE. A single thread simulates the default load behaviour while ROWS_NUM < 1M will default to a single thread for simplicity. Higher number of threads >=2 evaluates the benefit of parallel loading for the HT.

Change-Id: I1e0f07bf93ae57ff54022dc145fc9a7defd44782

3 months agoHIVE-25845: Support ColumnIndexes for Parq files (#3094)
rbalamohan [Mon, 14 Mar 2022 04:54:44 +0000 (10:24 +0530)] 
HIVE-25845: Support ColumnIndexes for Parq files (#3094)

3 months agoHIVE-25990: Optimise multiple copies in case of CTAS in external tables for Object...
Ayush Saxena [Mon, 14 Mar 2022 04:48:05 +0000 (10:18 +0530)] 
HIVE-25990: Optimise multiple copies in case of CTAS in external tables for Object stores. (#3058). (Ayush Saxena, reviewed by  Rajesh Balamohan)

3 months agoHIVE-26001: LlapServiceDriver: forward hiveconf from commandline to AsyncTaskCreateUd...
Bodor Laszlo [Sat, 12 Mar 2022 15:05:31 +0000 (16:05 +0100)] 
HIVE-26001: LlapServiceDriver: forward hiveconf from commandline to AsyncTaskCreateUdfFile (#3074) (Laszlo Bodor reviewed by Ramesh Kumar)

3 months agoHIVE-25963: Create temporary table with not null constraint gets converted to externa...
Sourabh Goyal [Wed, 16 Feb 2022 23:51:23 +0000 (15:51 -0800)] 
HIVE-25963: Create temporary table with not null constraint gets converted to external table(Sourabh Goyal via Yu-Wen Lai and Naveen Gangam)

Change-Id: I05e6d1e2504d3695c8cfea8c87c6f4ed5dfc92e0

3 months agoHIVE-26017 : Insert with partition value containing colon and space is creating parti...
mahesh kumar behera [Thu, 10 Mar 2022 16:18:34 +0000 (21:48 +0530)] 
HIVE-26017 : Insert with partition value containing colon and space is creating partition having wrong value (Mahesh Kumar Behera, reviewed by Ramesh Kumar Thangarajan)

3 months agoHIVE-25989: HBase CTLT failure rollback should not drop the underlying HBase table...
Marton Bod [Thu, 10 Mar 2022 11:09:54 +0000 (12:09 +0100)] 
HIVE-25989: HBase CTLT failure rollback should not drop the underlying HBase table (#3076) (Marton Bod, reviewed by Peter Vary)

3 months agoHIVE-25943: Introduce compaction cleaner failed attempts threshold (Laszlo Vegh,...
veghlaci05 [Thu, 10 Mar 2022 11:06:23 +0000 (12:06 +0100)] 
HIVE-25943: Introduce compaction cleaner failed attempts threshold (Laszlo Vegh, reviewed by Karen Coppage and Denys Kuzmenko)

Closes #3034.

3 months agoHIVE-25645: Query-based compaction doesn't work when partition column type is boolean...
veghlaci05 [Thu, 10 Mar 2022 11:03:19 +0000 (12:03 +0100)] 
HIVE-25645: Query-based compaction doesn't work when partition column type is boolean (Laszlo Vegh, reviewed by Karen Coppage)

Closes #3079.

3 months agoHIVE-26022: Error: ORA-00904 when initializing metastore schema in Oracle (Stamatis...
Stamatis Zampetakis [Wed, 9 Mar 2022 16:08:38 +0000 (17:08 +0100)] 
HIVE-26022: Error: ORA-00904 when initializing metastore schema in Oracle (Stamatis Zampetakis, reviewed by Laszlo Bodor)

Closes #3088

3 months agoHIVE-25935: Cleanup IMetaStoreClient#getPartitionsByNames APIs (Peter Vary reviewed...
pvary [Thu, 10 Mar 2022 09:03:45 +0000 (10:03 +0100)] 
HIVE-25935: Cleanup IMetaStoreClient#getPartitionsByNames APIs (Peter Vary reviewed by Zhihua Deng) (#3072)

3 months agoHIVE-25688: Addendum (Denys Kuzmenko, reviewed by Karen Coppage)
Denys Kuzmenko [Wed, 9 Mar 2022 14:10:16 +0000 (16:10 +0200)] 
HIVE-25688: Addendum (Denys Kuzmenko, reviewed by Karen Coppage)

Closes #3049

3 months agoHIVE-25977: Enhance Compaction Cleaner to skip when there is nothing to do #2 (#2971...
Zoltan Haindrich [Wed, 9 Mar 2022 12:04:11 +0000 (13:04 +0100)] 
HIVE-25977: Enhance Compaction Cleaner to skip when there is nothing to do #2 (#2971) (Zoltan Haindrich reviewed by Karen Coppage and Denys Kuzmenko)

3 months agoHIVE-26010: Compactions not picked up by cleaner if CQ_COMMIT_TIME is null and delaye...
veghlaci05 [Wed, 9 Mar 2022 08:28:55 +0000 (09:28 +0100)] 
HIVE-26010: Compactions not picked up by cleaner if CQ_COMMIT_TIME is null and delayed cleaning is enabled (Laszlo Vegh, reviewed by Karen Coppage)

Closes #3080.

3 months agoHIVE-25896: Remove getThreadId from IHMSHandler (#3017) (Zhihua Deng, reviewed by...
dengzh [Wed, 9 Mar 2022 02:12:41 +0000 (10:12 +0800)] 
HIVE-25896: Remove getThreadId from IHMSHandler (#3017) (Zhihua Deng, reviewed by Peter Vary and Karen Coppage)

3 months agoHIVE-26000: DirectSQL to pruning partitions fails with postgres backend for Skewed...
Naresh PR [Wed, 2 Mar 2022 21:31:47 +0000 (13:31 -0800)] 
HIVE-26000: DirectSQL to pruning partitions fails with postgres backend for Skewed-Partition tables (Naresh PR via Zoltan and Naveen)

3 months agoHIVE-25971: Closing the thread pool created for async cache (#3078)
Shailesh Gupta [Sun, 6 Mar 2022 22:20:32 +0000 (03:50 +0530)] 
HIVE-25971: Closing the thread pool created for async cache (#3078)

(cherry picked from commit a6153179f1f981bdce6adf5d29c3707d3367bbac)

Co-authored-by: Shailesh Gupta <shilgu@amazon.com>
3 months agoHIVE-25988: CreateTableEvent should have database object as one of the hive privilege...
saihemanth [Fri, 25 Feb 2022 18:45:11 +0000 (00:15 +0530)] 
HIVE-25988: CreateTableEvent should have database object as one of the hive privilege object(Saihemanth Gantasala via Naveen Gangam)

3 months agoHIVE-25894: Table migration to Iceberg doesn't remove HMS partitions (Peter Vary...
pvary [Fri, 4 Mar 2022 13:37:47 +0000 (14:37 +0100)] 
HIVE-25894: Table migration to Iceberg doesn't remove HMS partitions (Peter Vary reviewed by Marton Bod and Laszlo Pinter) (#3061)

3 months agoHIVE-25975: Optimize ClusteredWriter for bucketed Iceberg tables (#3060) (Adam Szita...
Adam Szita [Fri, 4 Mar 2022 11:27:26 +0000 (12:27 +0100)] 
HIVE-25975: Optimize ClusteredWriter for bucketed Iceberg tables (#3060) (Adam Szita, reviewed by Peter Vary and Marton Bod)

3 months agoHIVE-26004: Upgrade Iceberg to 0.13.1 (#3077) (Marton Bod, reviewed by Peter Vary)
Marton Bod [Fri, 4 Mar 2022 08:48:41 +0000 (09:48 +0100)] 
HIVE-26004: Upgrade Iceberg to 0.13.1 (#3077) (Marton Bod, reviewed by Peter Vary)

3 months agoHIVE-25997: Fix release source packaging (Zoltan Haindrich and Peter Vary, reviewed...
pvary [Thu, 3 Mar 2022 14:48:43 +0000 (15:48 +0100)] 
HIVE-25997: Fix release source packaging (Zoltan Haindrich and Peter Vary, reviewed by Stamatis Zampetakis) (#3067)

3 months agoHIVE-24390: Spelling common (#2810) (Josh Soref reviewed by Zoltan Haindrich)
Josh Soref [Thu, 3 Mar 2022 12:33:33 +0000 (07:33 -0500)] 
HIVE-24390: Spelling common (#2810) (Josh Soref reviewed by Zoltan Haindrich)

3 months agoHIVE-25986: Statement id is incorrect in case of load in path to MM table (Antal...
Antal Sinkovits [Wed, 2 Mar 2022 14:48:35 +0000 (15:48 +0100)] 
HIVE-25986: Statement id is incorrect in case of load in path to MM table (Antal Sinkovits, reviewed by Peter Vary)

Closes #3055

3 months agoHIVE-24949: Fail to rename a partition with customized catalog (#2910) (Zhihua Deng...
dengzh [Wed, 2 Mar 2022 01:43:29 +0000 (09:43 +0800)] 
HIVE-24949: Fail to rename a partition with customized catalog (#2910) (Zhihua Deng reviewed by Zoltan Haindrich)

3 months agoHIVE-25665: Checkstyle LGPL files must not be in the release sources/binaries (Peter...
pvary [Tue, 1 Mar 2022 22:16:17 +0000 (23:16 +0100)] 
HIVE-25665: Checkstyle LGPL files must not be in the release sources/binaries (Peter Vary reviewed by Zoltan Haindrich) (#3063)

3 months agoHIVE-23556: Support hive.metastore.limit.partition.request for get_partitions_ps...
zhuyan [Tue, 1 Mar 2022 12:00:11 +0000 (21:00 +0900)] 
HIVE-23556: Support hive.metastore.limit.partition.request for get_partitions_ps (#3021) (Benny Zhu reviewed by Zoltan Haindrich)

3 months agoHIVE-25981: Avoid checking for archived parts in analyze table. (#3052). (Ayush Saxen...
Ayush Saxena [Tue, 1 Mar 2022 04:08:37 +0000 (09:38 +0530)] 
HIVE-25981: Avoid checking for archived parts in analyze table. (#3052). (Ayush Saxena, reviewed by  Rajesh Balamohan and Peter Vary)

3 months agoHIVE-25178 [addendum]: Enable hive.load.dynamic.partitions.scan.specific.partitions...
Sourabh Goyal [Thu, 17 Feb 2022 18:49:24 +0000 (10:49 -0800)] 
HIVE-25178 [addendum]: Enable hive.load.dynamic.partitions.scan.specific.partitions by default

Enabling the property hive.load.dynamic.partitions.scan.specific.partitions othewise a
user sees significant performance degradation in insert/insert overwrite queries.

3 months agoHIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagiri via...
Naveen Gangam [Fri, 25 Feb 2022 18:51:38 +0000 (13:51 -0500)] 
HIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagiri via Naveen Gangam)

This reverts commit 2a1a73f665eee497ebdb0745ab2c31c1614de017.

3 months agoHIVE-25970: Missing messages in HS2 operation logs (Stamatis Zampetakis, reviewed...
Stamatis Zampetakis [Wed, 23 Feb 2022 15:02:27 +0000 (16:02 +0100)] 
HIVE-25970: Missing messages in HS2 operation logs (Stamatis Zampetakis, reviewed by Zoltan Haindrich)

Revert HIVE-22753 (commit 6a5c0cd04a2e88a545a96d10a942c86b2be18daa).

Preventing the creation of the appender (by returning null) leads to
the message triggering the creation to be lost forever. Moreover, the
memory leak that was observed in HIVE-22753 is no longer feasible with
the fix for HIVE-24590 so the revert is completely safe.

Closes #3048

4 months agoHIVE-25979: Order of Lineage is flaky in qtest output (Krisztian Kasa, reviewed by...
Krisztian Kasa [Fri, 25 Feb 2022 13:10:05 +0000 (14:10 +0100)] 
HIVE-25979: Order of Lineage is flaky in qtest output (Krisztian Kasa, reviewed by Zoltan Haindrich)

4 months agoRevert "HIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagi...
Naveen Gangam [Fri, 25 Feb 2022 04:50:26 +0000 (23:50 -0500)] 
Revert "HIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagiri via Naveen Gangam)"

This reverts commit fe14d020470b8494f01ab28afad8ff9abfee0665.

4 months agoHIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagiri via...
Abhay Chennagiri [Mon, 29 Nov 2021 17:49:45 +0000 (09:49 -0800)] 
HIVE-25750: [standalone beeline] Fixing the bug in HIVE-24348.(Abhay Chennagiri via Naveen Gangam)
Resolved the issues with the file mode on. No dependency on JobConf

4 months agoHIVE-25973: Fix JsonReporter and JsonFileMetricsReporter writing the same file (...
Viktor Csomor [Thu, 24 Feb 2022 08:00:09 +0000 (09:00 +0100)] 
HIVE-25973: Fix JsonReporter and JsonFileMetricsReporter writing the same file (#3047) (Viktor Csomor, reviewed by Laszlo Pinter)

4 months agoHIVE-25958: Optimise BasicStatsNoJobTask. (#3037). (Ayush Saxena, reviewed by Rajesh...
Ayush Saxena [Wed, 23 Feb 2022 15:59:17 +0000 (21:29 +0530)] 
HIVE-25958: Optimise BasicStatsNoJobTask. (#3037). (Ayush Saxena, reviewed by  Rajesh Balamohan)

4 months agoHIVE-25964: Create iceberg table with ranger authorization failed with storage URI...
Butao Zhang [Wed, 23 Feb 2022 09:56:30 +0000 (17:56 +0800)] 
HIVE-25964: Create iceberg table with ranger authorization failed with storage URI NullPointerException (#3038) (Butao Zhang, reviewed by Marton Bod)

4 months agoHIVE-25967: Prevent residual expressions from getting serialized in Iceberg splits...
Adam Szita [Tue, 22 Feb 2022 15:12:51 +0000 (16:12 +0100)] 
HIVE-25967: Prevent residual expressions from getting serialized in Iceberg splits (#3041) (Adam Szita, reviewed by Peter Vary)

4 months agoHIVE-25874: Slow filter evaluation of nest struct fields in vectorized executions...
Zoltan Haindrich [Tue, 22 Feb 2022 14:49:35 +0000 (15:49 +0100)] 
HIVE-25874: Slow filter evaluation of nest struct fields in vectorized executions (#2952) (Zoltan Haindrich reviewed by Krisztian Kasa)

4 months agoHIVE-25960: Fix S3a recursive listing logic. (#3031). (Ayush Saxena reviewed by Laszl...
Ayush Saxena [Tue, 22 Feb 2022 14:14:02 +0000 (19:44 +0530)] 
HIVE-25960: Fix S3a recursive listing logic. (#3031). (Ayush Saxena reviewed by Laszlo Bodor)

4 months agoHIVE-25938: Print excluded rules from CBO (Alessandro Solimando, reviewed by Stamatis...
Alessandro Solimando [Tue, 8 Feb 2022 16:09:19 +0000 (17:09 +0100)] 
HIVE-25938: Print excluded rules from CBO (Alessandro Solimando, reviewed by Stamatis Zampetakis, John Sherman)

Closes #3011

4 months agoHIVE-25959: Expose Compaction Observability delta metrics using the JsonReporter...
Viktor Csomor [Tue, 22 Feb 2022 10:38:10 +0000 (11:38 +0100)] 
HIVE-25959: Expose Compaction Observability delta metrics using the JsonReporter (Viktor Csomor, reviewed by Karen Coppage)

- Custom MapMetrics has been included into the MetricsRegistry.
- The MetricsRegistry will call the listeners for the built-in metrics. Since the MapMetrics is not a built-in one it is causing an IllegalArgument Exception. This exception is handled.
- A new Jackson JSON module has been added (MapCapableJsonMetricsModule) to add serialization of the `mbeans`
- MapCapableJsonMetricsModule has been registered to the JsonReporter
- AcidMetricsService now leveraging both the on the new metrics and the MetricsMBeanImpl+JMX

Closes #3032.

4 months agoHIVE-25918: Invalid stats after multi inserting into the same partition (Krisztian...
Krisztian Kasa [Tue, 22 Feb 2022 05:36:24 +0000 (06:36 +0100)] 
HIVE-25918: Invalid stats after multi inserting into the same partition (Krisztian Kasa, reviewed by Zoltan Haindrich)

4 months agoHIVE-25149: Support parallel load for Fast HT implementations (#3029) (Panagiotis...
Panagiotis Garefalakis [Fri, 18 Feb 2022 19:08:03 +0000 (11:08 -0800)] 
HIVE-25149: Support parallel load for Fast HT implementations (#3029) (Panagiotis Garefalakis, reviewed by Ramesh Kumar)

* HIVE-25149: Support parallel load for Fast HT implementations
* Introducing new HiveConf controlling the number of threads used in parallel HT load for MJ
* Extending VectorMapJoinFastHashTableLoader to support parallel HT loading

4 months agoHIVE-25947: Compactor job queue cannot be set per table via compactor.mapred.job...
Stamatis Zampetakis [Mon, 31 Jan 2022 10:45:17 +0000 (11:45 +0100)] 
HIVE-25947: Compactor job queue cannot be set per table via compactor.mapred.job.queue.name (Stamatis Zampetakis, reviewed by Alessandro Solimando, Denys Kuzmenko)

Adapt the MR compactor to accept all the properties below:
* compactor.mapred.job.queue.name
* compactor.mapreduce.job.queuename
* compactor.hive.compactor.job.queue

for specifying the job queue per table and per compaction. The change
restores backward compatibility and also enables the use of the non
deprecated MR properties.

Add unit tests defining and guarding the precedence among the
aforementioned properties and the different granularity at which a
queue can be defined.

Closes #3027

4 months agoHIVE-25957: Fix password based authentication with SAML enabled (#3028) (Yu-Wen Lai...
Yu-Wen [Thu, 17 Feb 2022 20:02:01 +0000 (12:02 -0800)] 
HIVE-25957: Fix password based authentication with SAML enabled (#3028) (Yu-Wen Lai, reviewed by Naveen Gangam)

* HIVE-25957: Fix password based authentication with SAML enabled

* Adds unit tests for AuthType

* Remove unused argument for getAuthHeader

4 months agoHIVE-21152: Rewrite if expression to case and recognize simple case as an if (#2791...
Zoltan Haindrich [Thu, 17 Feb 2022 17:12:06 +0000 (18:12 +0100)] 
HIVE-21152: Rewrite if expression to case and recognize simple case as an if (#2791) (Zoltan Haindrich reviewed by Krisztian Kasa)

4 months agoHIVE-25715: Provide nightly builds (#3013) (Zoltan Haindrich reviewed by Krisztian...
Zoltan Haindrich [Thu, 17 Feb 2022 17:06:19 +0000 (18:06 +0100)] 
HIVE-25715: Provide nightly builds (#3013) (Zoltan Haindrich reviewed by Krisztian Kasa)

4 months agoHIVE-25915: Query based MINOR compaction fails with NPE if the data is loaded into...
veghlaci05 [Thu, 17 Feb 2022 13:51:30 +0000 (14:51 +0100)] 
HIVE-25915: Query based MINOR compaction fails with NPE if the data is loaded into the ACID table (Laszlo Vegh, reviewed by Denys Kuzmenko)

Closes #3000

4 months agoHIVE-25951: HiveRelMdPredicates extends RelMdPredicates (Alessandro Solimando, review...
Alessandro Solimando [Thu, 17 Feb 2022 12:44:54 +0000 (13:44 +0100)] 
HIVE-25951: HiveRelMdPredicates extends RelMdPredicates (Alessandro Solimando, reviewed by Krisztian Kasa)

4 months agoHIVE-25884: Improve rule description for rules defined as subclasses (Alessandro...
Alessandro Solimando [Thu, 17 Feb 2022 12:25:36 +0000 (13:25 +0100)] 
HIVE-25884: Improve rule description for rules defined as subclasses (Alessandro Solimando reviewed by Zoltan Haindrich, Krisztian Kasa)

4 months agoHIVE-25961: Altering partition specification parameters for Iceberg tables are not...
pvary [Thu, 17 Feb 2022 06:28:08 +0000 (07:28 +0100)] 
HIVE-25961: Altering partition specification parameters for Iceberg tables are not working (Peter Vary reviewed by Laszlo Pinter) (#3035)

4 months agoHIVE-25944: Format pom.xml-s (#3010) (Zoltan Haindrich reviewed by Zhihua Deng)
Zoltan Haindrich [Wed, 16 Feb 2022 16:18:13 +0000 (17:18 +0100)] 
HIVE-25944: Format pom.xml-s (#3010) (Zoltan Haindrich reviewed by Zhihua Deng)

4 months agoHIVE-25747: Make a cost base decision when rebuilding materialized views (Krisztian...
Krisztian Kasa [Wed, 16 Feb 2022 08:14:53 +0000 (09:14 +0100)] 
HIVE-25747: Make a cost base decision when rebuilding materialized views (Krisztian Kasa, reviewed by Zoltan Haindrich, Aman Sinha)

4 months agoHIVE-25955: Partitioned tables migrated to Iceberg aren't cached in LLAP (#3026)...
Adam Szita [Tue, 15 Feb 2022 07:49:58 +0000 (08:49 +0100)] 
HIVE-25955: Partitioned tables migrated to Iceberg aren't cached in LLAP (#3026) (Adam Szita, reviewed by Peter Vary)

4 months agoHIVE-25912: Drop external table throw NPE if the location set to ROOT (Fachuan Bai...
白发川(惊帆) [Mon, 14 Feb 2022 06:48:47 +0000 (14:48 +0800)] 
HIVE-25912: Drop external table throw NPE if the location set to ROOT (Fachuan Bai reviewed by Peter Vary) (#3009)

Closes #3009

4 months agoHIVE-25895: Bootstrap tables in table_diff during Incremental Load. (#2980). (Ayush...
Ayush Saxena [Mon, 14 Feb 2022 06:20:20 +0000 (11:50 +0530)] 
HIVE-25895: Bootstrap tables in table_diff during Incremental Load. (#2980). (Ayush Saxena, reviewed by Pravin Kumar Sinha)

4 months agoHIVE-25942: Upgrade commons-io version to 2.8.0 due to CVE-2021-29425. (#3008) (Syed...
Syed Shameerur Rahman [Sun, 13 Feb 2022 11:30:59 +0000 (17:00 +0530)] 
HIVE-25942: Upgrade commons-io version to 2.8.0 due to CVE-2021-29425. (#3008) (Syed Shameerur Rahman reviewed by Zoltan Haindrich)

4 months agoHIVE-25945: Upgrade H2 database version to 2.1.210 (Stamatis Zampetakis, reviewed...
Stamatis Zampetakis [Wed, 9 Feb 2022 21:50:18 +0000 (22:50 +0100)] 
HIVE-25945: Upgrade H2 database version to 2.1.210 (Stamatis Zampetakis, reviewed by Zhihua Deng)

The 1.3.166 version suffers from the following security vulnerabilities:
https://nvd.nist.gov/vuln/detail/CVE-2021-42392
https://nvd.nist.gov/vuln/detail/CVE-2022-23221

In the project, we use H2 only for testing purposes (inside the
jdbc-handler module) thus the H2 binaries are not present in the
runtime classpath thus these CVEs do not pose a problem for Hive or its
users.

Closes #3012

4 months agoHIVE-25583: Support parallel load for HastTables - Interfaces (#2999) (Panagiotis...
Panagiotis Garefalakis [Thu, 10 Feb 2022 18:14:12 +0000 (20:14 +0200)] 
HIVE-25583: Support parallel load for HastTables - Interfaces (#2999) (Panagiotis Garefalakis reviewed by Ramesh Kumar)

* Introducing VectorMapJoinFastHashTableContainerBase class that implements VectorMapJoinHashTable
* Each VectorMapJoinFastStringHashMapContainer is a singleton that contains an array of HashTables (1 or more)
* VectorMapJoinFastTableContainer now initializes VectorMapJoinFastHashTableContainers instead of HTs directly

Change-Id: I14f71dab3859b75d822914966151548793998c39

4 months agoHIVE-25892: Group HMSHandler's thread locals into a single context (#2967) (Zhihua...
dengzh [Thu, 10 Feb 2022 13:12:40 +0000 (21:12 +0800)] 
HIVE-25892: Group HMSHandler's thread locals into a single context (#2967) (Zhihua Deng, reviewed by Peter Vary and Zoltan Haindrich)

4 months agoHIVE-25918: Invalid stats after multi inserting into the same partition (Krisztian...
Krisztian Kasa [Wed, 9 Feb 2022 16:50:33 +0000 (17:50 +0100)] 
HIVE-25918: Invalid stats after multi inserting into the same partition (Krisztian Kasa, reviewed by Stamatis Zampetakis)

4 months agoHIVE-25937: Create view fails when definition contains a materialized view definition...
Krisztian Kasa [Wed, 9 Feb 2022 16:46:38 +0000 (17:46 +0100)] 
HIVE-25937: Create view fails when definition contains a materialized view definition (Krisztian Kasa, reviewed by Zoltan Haindrich)

4 months agoHIVE-25919: ClassCastException when pushing boolean column predicate in HBaseStorageH...
Stamatis Zampetakis [Thu, 3 Feb 2022 16:16:28 +0000 (17:16 +0100)] 
HIVE-25919: ClassCastException when pushing boolean column predicate in HBaseStorageHandler (Stamatis Zampetakis, reviewed by Laszlo Bodor)

The predicate analyzer can only handle expressions of the form:
i) col comp_op constant
ii) constant comp_op col
and these are always ExprNodeGenericFuncDesc.

The new if clause ensures that we are never going to handle anything
that is not supported.

Closes #2996

4 months agoHIVE-25898: Compaction txn heartbeating after Worker timeout (Laszlo Vegh, reviewed... 2940/head
veghlaci05 [Tue, 8 Feb 2022 14:11:49 +0000 (15:11 +0100)] 
HIVE-25898: Compaction txn heartbeating after Worker timeout (Laszlo Vegh, reviewed by Denys Kuzmenko, Stamatis Zampetakis)

Closes #2981

4 months agoHIVE-25766 java.util.NoSuchElementException in HiveFilterProjectTransposeRule if...
Alessandro Solimando [Mon, 7 Feb 2022 09:03:24 +0000 (10:03 +0100)] 
HIVE-25766  java.util.NoSuchElementException in HiveFilterProjectTransposeRule if predicate has no InputRef (Alessandro Solimando reviewed by Zoltan Haindrich)

4 months agoHIVE-25926: Move all logging from AcidMetricService to AcidMetricLogger (Viktor Csomo...
Viktor Csomor [Mon, 7 Feb 2022 08:37:53 +0000 (09:37 +0100)] 
HIVE-25926: Move all logging from AcidMetricService to AcidMetricLogger (Viktor Csomor, reviewed by Karen Coppage)

The common logic required by the `AcidMetricLogger` and the `AcidMetricService` had been extracted to a package-private component `AcidMetricData`.
This change enabled to move the logging from AcidMetricService to AcidMetricLogger.

Added methods:
- logMultipleWorkerVersions
- logFailedCompactionsPercentage
- logOldestInitiatorAge

Tests added.

Closes #2995.

4 months agoHIVE-25927: Fix DataWritableReadSupport (#2998)
rbalamohan [Mon, 7 Feb 2022 00:42:50 +0000 (06:12 +0530)] 
HIVE-25927: Fix DataWritableReadSupport (#2998)

4 months agoHIVE-25706: ShuffleHandler: Make sure of properly releasing netty reference counted...
Bodor Laszlo [Sat, 5 Feb 2022 16:56:36 +0000 (17:56 +0100)] 
HIVE-25706: ShuffleHandler: Make sure of properly releasing netty reference counted objects (#2794) (Laszlo Bodor reviewed by Panagiotis Garefalakis)

4 months agoHIVE-25772: Use ClusteredWriter when writing to Iceberg tables (#2842) (Adam Szita...
Adam Szita [Fri, 4 Feb 2022 14:25:00 +0000 (15:25 +0100)] 
HIVE-25772: Use ClusteredWriter when writing to Iceberg tables (#2842) (Adam Szita, reviewed by Marton Bod)

4 months agoHIVE-25916: Optimise updateCompactionMetricsData (#2989) (Laszlo Pinter, reviewed...
László Pintér [Fri, 4 Feb 2022 11:31:48 +0000 (12:31 +0100)] 
HIVE-25916: Optimise updateCompactionMetricsData (#2989) (Laszlo Pinter, reviewed by Karen Coppage)

4 months agoHIVE-25922: Upgrade Iceberg dependency to 0.13.0 (#2994) (Adam Szita, reviewed by...
Adam Szita [Thu, 3 Feb 2022 13:30:51 +0000 (14:30 +0100)] 
HIVE-25922: Upgrade Iceberg dependency to 0.13.0 (#2994) (Adam Szita, reviewed by Marton Bod)

4 months agoHIVE-25920: Bump Xerce2 to 2.12.2. (#2992). (Takanobu Asanuma, reviewed by Ayush...
Takanobu Asanuma [Thu, 3 Feb 2022 11:51:40 +0000 (20:51 +0900)] 
HIVE-25920: Bump Xerce2 to 2.12.2. (#2992). (Takanobu Asanuma, reviewed by Ayush Saxena)

4 months agoHIVE-25875: Hive support for parallel authentication types LDAP/SAML in http mode...
saihemanth [Wed, 19 Jan 2022 18:16:21 +0000 (23:46 +0530)] 
HIVE-25875: Hive support for parallel authentication types LDAP/SAML in http mode (Saihemanth Gantasala reviewed by Naveen Gangam)

4 months agoRevert "HIVE-25887 - Add external_table_concatenate.q to testconfiguration.properties...
Stamatis Zampetakis [Wed, 2 Feb 2022 10:56:52 +0000 (11:56 +0100)] 
Revert "HIVE-25887 - Add external_table_concatenate.q to testconfiguration.properties. (#2959) (Harish Jaiprakash, reviewed by Naveen Gangam)"

This reverts commit 28dc8c17a49f861ec03689369c981097b0daa5d6.

4 months agoHIVE-25746: Fix of Compaction Failure Counter counted incorrectly (Viktor Csomor...
Viktor Csomor [Tue, 1 Feb 2022 15:07:14 +0000 (16:07 +0100)] 
HIVE-25746: Fix of Compaction Failure Counter counted incorrectly (Viktor Csomor, reviewed by Denys Kuzmenko and Karen Coppage)

Fixing compaction_initiator_failure_counter/compaction_cleaner_failure_counter logic in the Initiator and Cleaner.
- The `exceptionally` closure made the `.allOf(..).join()` call complete always without any errors
- The counting logic of the compaction_initiator_failure_counter moved to the `scheduleCompactionIfRequired` method
- The counting logic of the compaction_cleaner_failure_counter moved to the `clean` method

Both the `scheduleCompactionIfRequired` and `clean` methods catching all the possible exception except the `MetaException` of the `txnHandler.markFailed()` call. Hence, the values only incremented if there were serious issues with the mark failed call.

Closes #2974.

4 months agoHIVE-25909: Add test for 'hive.default.nulls.last' property for windows with ordering...
Alessandro Solimando [Thu, 27 Jan 2022 10:32:23 +0000 (11:32 +0100)] 
HIVE-25909: Add test for 'hive.default.nulls.last' property for windows with ordering (Alessandro Solimando, reviewed by Stamatis Zampetakis)

Closes #2979

4 months agoHIVE-25902: Vectorized reading of Parquet tables via Iceberg (Adam Szita, reviewed...
Adam Szita [Tue, 1 Feb 2022 10:24:58 +0000 (11:24 +0100)] 
HIVE-25902: Vectorized reading of Parquet tables via Iceberg (Adam Szita, reviewed by Marton Bod and Peter Vary)

4 months agoHIVE-25893: NPE when reading Parquet data because ColumnVector isNull[] is not update...
Soumyakanti Das [Tue, 1 Feb 2022 07:09:20 +0000 (23:09 -0800)] 
HIVE-25893: NPE when reading Parquet data because ColumnVector isNull[] is not updated (Soumyakanti Das, reviewed by Krisztian Kasa)

4 months agoHIVE-25819: Track event id on target cluster with respect to source cluster. (#2890...
Ayush Saxena [Tue, 1 Feb 2022 04:10:33 +0000 (09:40 +0530)] 
HIVE-25819: Track event id on target cluster with respect to source cluster. (#2890). (Ayush Saxena, reviewed by Pravin Kumar Sinha)

4 months agoHIVE-25917: Use default value for 'hive.default.nulls.last' when config is not availa...
Alessandro Solimando [Thu, 27 Jan 2022 09:35:57 +0000 (10:35 +0100)] 
HIVE-25917: Use default value for 'hive.default.nulls.last' when config is not available (Alessandro Solimando, reviewed by Stamatis Zampetakis)

Closes #2990