hive.git
8 hours agoHIVE-26349: TestOperatorCmp/TestReOptimization fail silently due to incompatible... master
Stamatis Zampetakis [Wed, 22 Jun 2022 12:19:30 +0000 (14:19 +0200)] 
HIVE-26349: TestOperatorCmp/TestReOptimization fail silently due to incompatible configuration (Stamatis Zampetakis, reviewed by Peter Vary, Ayush Saxena)

Closes #3398

39 hours agoHIVE-26057: Clean useless methods in QueryWrapper (Zhihua Deng, reviewed by Peter...
dengzh [Tue, 5 Jul 2022 01:52:44 +0000 (09:52 +0800)] 
HIVE-26057: Clean useless methods in QueryWrapper (Zhihua Deng, reviewed by Peter Vary)

Closes #3124

4 days agoHIVE-26182: Some improvements to make DPP more debuggable (#3257) (Laszlo Bodor revie...
Bodor Laszlo [Sat, 2 Jul 2022 06:34:13 +0000 (08:34 +0200)] 
HIVE-26182: Some improvements to make DPP more debuggable (#3257) (Laszlo Bodor reviewed by Rajesh Balamohan)

5 days agoHIVE-26355: Column compare should be case insensitive for name (Wechar Yu reviewed...
Wechar Yu [Fri, 1 Jul 2022 05:28:02 +0000 (13:28 +0800)] 
HIVE-26355: Column compare should be case insensitive for name (Wechar Yu reviewed by Peter Vary) (#3406)

6 days agoHIVE-26365: Remove column statistics collection task from merge statement plan (Krisz...
Krisztian Kasa [Thu, 30 Jun 2022 12:22:28 +0000 (14:22 +0200)] 
HIVE-26365: Remove column statistics collection task from merge statement plan (Krisztian Kasa, reviewed by Peter Vary, Alessandro Solimando, Aman Sinha)

7 days agoHIVE-26242: Compaction heartbeater improvements (Laszlo Vegh, reviewed by Karen Coppa...
veghlaci05 [Wed, 29 Jun 2022 11:16:59 +0000 (13:16 +0200)] 
HIVE-26242: Compaction heartbeater improvements (Laszlo Vegh, reviewed by Karen Coppage, Denys Kuzmenko)

Closes #3303

7 days agoHIVE-25976: Cleaner may remove files being accessed from a fetch-task-converted reade...
veghlaci05 [Wed, 29 Jun 2022 09:52:36 +0000 (11:52 +0200)] 
HIVE-25976: Cleaner may remove files being accessed from a fetch-task-converted reader (Laszlo Vegh, reviewed by Peter Vary, Krisztian Kasa)

7 days agoDisable flaky test
Peter Vary [Wed, 29 Jun 2022 07:32:45 +0000 (09:32 +0200)] 
Disable flaky test

7 days agoHIVE-26358: Querying metadata tables does not work for Iceberg tables using HADOOP_TA...
pvary [Wed, 29 Jun 2022 07:16:52 +0000 (09:16 +0200)] 
HIVE-26358: Querying metadata tables does not work for Iceberg tables using HADOOP_TABLE (Peter Vary reviewed by Laszlo Pinter) (#3408)

7 days agoHIVE-26361: Merge statement compile fails if split update is off (Krisztian Kasa...
Krisztian Kasa [Wed, 29 Jun 2022 05:13:24 +0000 (07:13 +0200)] 
HIVE-26361: Merge statement compile fails if split update is off (Krisztian Kasa, reviewed by Peter Vary)

7 days agoHIVE-26319: Iceberg integration: Perform update split early (Krisztian Kasa, reviewed...
Krisztian Kasa [Wed, 29 Jun 2022 05:08:18 +0000 (07:08 +0200)] 
HIVE-26319: Iceberg integration: Perform update split early (Krisztian Kasa, reviewed by Peter Vary)

8 days agoHIVE-26354: Support expiring snapshots on iceberg table (Peter Vary reviewed by Laszl...
pvary [Tue, 28 Jun 2022 09:55:06 +0000 (11:55 +0200)] 
HIVE-26354: Support expiring snapshots on iceberg table (Peter Vary reviewed by Laszlo Pinter) (#3401)

8 days agoHIVE-26352: Tez queue access check fails with GSS Exception on Compaction (Laszlo...
veghlaci05 [Tue, 28 Jun 2022 08:59:49 +0000 (10:59 +0200)] 
HIVE-26352: Tez queue access check fails with GSS Exception on Compaction (Laszlo Vegh, reviewed by Denys Kuzmenko)

Closes #3402

8 days agoHIVE-26265: Option to filter out Txn events during replication. (Francis Pang reviewe...
cmunkey [Tue, 28 Jun 2022 07:48:22 +0000 (00:48 -0700)] 
HIVE-26265: Option to filter out Txn events during replication. (Francis Pang reviewed by Peter Vary) (#3365)

Purpose: Currently, all Txn events OpenTxn, CommitTxn, and RollbackTxn are included in the REPL DUMP, even when the transcation does not involve the database being dumped
(replicated). These events are unnecessary and result is excessive space required for the dump, as well as increasing work that results from these events being replayed
during REPL LOAD.

Solution proposed: To reduce this unnecessary space and work, added the hive.repl.filter.transactions configuration property. When set to "true", extra Txn events
will be filtered out as follows: CommitTxn and RollbackTxn are included in the REPL DUMP only if the transaction referenced had a corresponding ALLOCATE_WRITE_ID event
that was dumped. OpenTxn is never dumped, and the OpenTxn event will be implcitly Opened when REPL LOAD processes the ALLOC_WRITE_ID event, since the ALLOC_WRITE_ID
contains the open transaction ids. The default setting is "false".

Co-authored-by: Francis Pang <fpang@cloudera.com>
8 days agoHIVE-26128: Enabling dynamic runtime filtering in Iceberg tables throws exception...
Adam Szita [Tue, 28 Jun 2022 07:45:50 +0000 (09:45 +0200)] 
HIVE-26128: Enabling dynamic runtime filtering in Iceberg tables throws exception at runtime (#3404) (Adam Szita, reviewed by Peter Vary)

11 days agoHIVE-25980: Reduce fs calls in HiveMetaStoreChecker.checkTable (Chiran Ravani reviewe...
Chiran Ravani [Sat, 25 Jun 2022 09:34:52 +0000 (05:34 -0400)] 
HIVE-25980: Reduce fs calls in HiveMetaStoreChecker.checkTable (Chiran Ravani reviewed by Syed Shameerur Rahman and Peter Vary)(#3053)

13 days agoHIVE-26021: Change integration tests under DBInstallBase to Checkin tests (Stamatis...
Stamatis Zampetakis [Mon, 20 Jun 2022 11:38:20 +0000 (13:38 +0200)] 
HIVE-26021: Change integration tests under DBInstallBase to Checkin tests (Stamatis Zampetakis, reviewed by Peter Vary)

Drop failsafe plugin since it is no longer needed.

Rename tests to better reflect their purpose (not integration tests
anymore) but more importantly to allow Jenkins splitTests step (
https://plugins.jenkins.io/parallel-test-executor/) to pick them up
automatically.

Extend test coverage to include mysql and mssql install/upgrade tests.

Closes #3399

13 days agoHIVE-26244: Implement locking for the concurrent CTAS (Simhadri Govindappa, reviewed...
SimhadriGovindappa [Thu, 23 Jun 2022 09:55:33 +0000 (15:25 +0530)] 
HIVE-26244: Implement locking for the concurrent CTAS (Simhadri Govindappa, reviewed by Denys Kuzmenko)

Closes #3307

2 weeks agoHIVE-26334: Remove misleading bucketing info from DESCRIBE FORMATTED output for Icebe...
pvary [Wed, 22 Jun 2022 13:24:05 +0000 (15:24 +0200)] 
HIVE-26334: Remove misleading bucketing info from DESCRIBE FORMATTED output for Iceberg tables (Peter Vary reviewed by Laszlo Pinter) (#3378)

2 weeks agoHIVE-26149: Addendum: Fix materialized views removal when non-default MetaStoreFilter...
Denys Kuzmenko [Wed, 22 Jun 2022 10:22:04 +0000 (12:22 +0200)] 
HIVE-26149: Addendum: Fix materialized views removal when non-default MetaStoreFilterHook is configured (Denys Kuzmenko, reviewed by Karen Coppage)

Closes #3395

2 weeks agoHIVE-25827: Parquet file footer is read multiple times, when multiple splits are...
Adam Szita [Wed, 22 Jun 2022 09:07:43 +0000 (11:07 +0200)] 
HIVE-25827: Parquet file footer is read multiple times, when multiple splits are created in same file (#3368) (Adam Szita, reviewed by Peter Vary)

2 weeks agoHIVE-26343: Disable TestWebHCatE2e test cause it fails
Stamatis Zampetakis [Mon, 20 Jun 2022 14:47:45 +0000 (16:47 +0200)] 
HIVE-26343: Disable TestWebHCatE2e test cause it fails

Closes #3390

2 weeks agoHIVE-26240: Add tests for Esri UDFs for Geospatial. (#3385). (Ayush Saxena, reviewed...
Ayush Saxena [Tue, 21 Jun 2022 06:31:20 +0000 (12:01 +0530)] 
HIVE-26240: Add tests for Esri UDFs for Geospatial. (#3385). (Ayush Saxena, reviewed by Mahesh Kumar Behera)

2 weeks agoHIVE-26341: Replication logs & metrics shows last repl id as 0 when number of events...
Haymant Mangla [Tue, 21 Jun 2022 04:37:09 +0000 (10:07 +0530)] 
HIVE-26341: Replication logs & metrics shows last repl id as 0 when number of events is zero (Haymant Mangla, reviewed by  Rajesh Balamohan and Mahesh Kumar Behera)

2 weeks agoHIVE-26208: Exception in Vectorization with Decimal64 to Decimal casting (#3344)
scarlin-cloudera [Mon, 20 Jun 2022 18:15:04 +0000 (11:15 -0700)] 
HIVE-26208: Exception in Vectorization with Decimal64 to Decimal casting (#3344)

The exception was happening because the code that casts the Decimal64 to
Decimal was not being added in filtered expressions.

Once this code was added, it caused a regression in check_constraint.q. The
reason for this was because we do not want to convert Decimal64 to Decimal
if the expression explicitly handles decimal64 types. A method was added
to these classes that will prevent the conversion in these cases.

2 weeks agoHIVE-26298: Selecting complex types on migrated iceberg table does not work. (addendu...
László Pintér [Mon, 20 Jun 2022 11:38:42 +0000 (13:38 +0200)] 
HIVE-26298: Selecting complex types on migrated iceberg table does not work. (addendum) (#3383) (Laszlo Pinter)

2 weeks agoHIVE-26310: Remove unused junit runners from test-utils module (Stamatis Zampetakis...
Stamatis Zampetakis [Fri, 10 Jun 2022 12:29:09 +0000 (14:29 +0200)] 
HIVE-26310: Remove unused junit runners from test-utils module (Stamatis Zampetakis, reviewed by Ayush Saxena)

Closes #3358

2 weeks agoHIVE-26338: Repl Dump should fail if source database does not exist (Haymant Mangla...
Haymant Mangla [Sat, 18 Jun 2022 07:17:00 +0000 (12:47 +0530)] 
HIVE-26338: Repl Dump should fail if source database does not exist (Haymant Mangla, reviewed by Denys Kuzmenko)

Closes #3380

2 weeks agoHIVE-26326: Support enabling background threads when failover is in progress (Haymant...
Haymant Mangla [Sat, 18 Jun 2022 07:15:07 +0000 (12:45 +0530)] 
HIVE-26326: Support enabling background threads when failover is in progress (Haymant Mangla, reviewed by Peter Vary)

Closes #3376

2 weeks agoHIVE-26340: Vectorized PTF operator fails if query has upper case window function...
Krisztian Kasa [Fri, 17 Jun 2022 10:58:29 +0000 (12:58 +0200)] 
HIVE-26340: Vectorized PTF operator fails if query has upper case window function (Krisztian Kasa, reviewed by Laszlo Bodor)

2 weeks agoHIVE-26324: Add one-row-table constraints on NOTIFICATION_SEQUENCE table (Sourabh...
Sourabh Badhya [Fri, 17 Jun 2022 07:56:58 +0000 (13:26 +0530)] 
HIVE-26324: Add one-row-table constraints on NOTIFICATION_SEQUENCE table (Sourabh Badhya, reviewed by Denys Kuzmenko)

Closes #3369

2 weeks agoHIVE-26177: Amendment: Initialize datanucleus compaction pool only on HMS Leader...
Denys Kuzmenko [Fri, 17 Jun 2022 07:43:33 +0000 (09:43 +0200)] 
HIVE-26177: Amendment: Initialize datanucleus compaction pool only on HMS Leader & pool sizing (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3372

2 weeks agoHIVE-26331: Use maven-surefire-plugin version consistently in standalone-metastore...
Stamatis Zampetakis [Wed, 15 Jun 2022 11:59:24 +0000 (13:59 +0200)] 
HIVE-26331: Use maven-surefire-plugin version consistently in standalone-metastore modules (Stamatis Zampetakis, reviewed by Zoltan Haindrich, Ayush Saxena)

Closes #3374

2 weeks agoHIVE-26316: Handle dangling open txns on both src & tgt in unplanned failover. (Hayma...
Haymant Mangla [Thu, 16 Jun 2022 09:41:22 +0000 (15:11 +0530)] 
HIVE-26316: Handle dangling open txns on both src & tgt in unplanned failover. (Haymant Mangla reviewed by Peter Vary) (#3367)

2 weeks agoHIVE-26298: Selecting complex types on migrated iceberg table does not work. (#3361...
László Pintér [Thu, 16 Jun 2022 06:10:53 +0000 (08:10 +0200)] 
HIVE-26298: Selecting complex types on migrated iceberg table does not work. (#3361) (Laszlo Pinter, reviewed by Peter Vary)

2 weeks agoHIVE-26321: Upgrade commons-io to 2.11.0 (Naveen Gangam) (#3370)
Naveen Gangam [Wed, 15 Jun 2022 20:28:02 +0000 (16:28 -0400)] 
HIVE-26321: Upgrade commons-io to 2.11.0 (Naveen Gangam) (#3370)

3 weeks agoHIVE-26269 Class cast exception when vectorization is enabled for certain case when...
Ramesh Kumar [Wed, 15 Jun 2022 17:31:44 +0000 (10:31 -0700)] 
HIVE-26269 Class cast exception when vectorization is enabled for certain case when cases (#3329)

3 weeks agoHIVE-26309: Remove Log4jConfig junit extension in favor of LoggerContextSource (Stama...
Stamatis Zampetakis [Thu, 9 Jun 2022 15:19:32 +0000 (17:19 +0200)] 
HIVE-26309: Remove Log4jConfig junit extension in favor of LoggerContextSource (Stamatis Zampetakis, reviewed by Alessandro Solimando, Laszlo Bodor)

Closes #3356

3 weeks agoHIVE-25733: Add check-spelling/check-spelling (#2809) (Josh Soref reviewed by Zoltan...
Peter Vary [Wed, 15 Jun 2022 08:48:01 +0000 (10:48 +0200)] 
HIVE-25733: Add check-spelling/check-spelling (#2809) (Josh Soref reviewed by Zoltan Haindrich) (Addendum)

3 weeks agoHIVE-26299: Drop data connector ifNotExists should not throw NoSuchObjectException...
Butao Zhang [Tue, 14 Jun 2022 19:20:59 +0000 (03:20 +0800)] 
HIVE-26299: Drop data connector ifNotExists should not throw NoSuchObjectException (Reviewed by Saihemanth) (#3350)

3 weeks agoHIVE-25879: MetaStoreDirectSql test query should not query the whole DBS table (...
mszurap [Tue, 14 Jun 2022 13:15:18 +0000 (15:15 +0200)] 
HIVE-25879: MetaStoreDirectSql test query should not query the whole DBS table (#3348) (Miklos Szurap reviewed by Zoltan Haindrich)

3 weeks agoHIVE-25503: Amendment: Null-safe equality comparison (Denys Kuzmenko, reviewed by...
Denys Kuzmenko [Tue, 14 Jun 2022 08:21:42 +0000 (10:21 +0200)] 
HIVE-25503: Amendment: Null-safe equality comparison (Denys Kuzmenko, reviewed by Karen Coppage, Laszlo Vegh)

Closes #3357

3 weeks agoHIVE-26280: Copy more data into COMPLETED_COMPACTIONS for better supportability ...
Karen Coppage [Tue, 14 Jun 2022 07:17:53 +0000 (09:17 +0200)] 
HIVE-26280: Copy more data into COMPLETED_COMPACTIONS for better supportability (Karen Coppage, reviewed by Denys Kuzmenko)

Closes #3340.

3 weeks agoHIVE-26142: Extend the hidden conf list with webui keystore pwd (Janos Kovacs, review...
Janos Kovacs [Tue, 14 Jun 2022 07:12:15 +0000 (09:12 +0200)] 
HIVE-26142: Extend the hidden conf list with webui keystore pwd (Janos Kovacs, reviewed by Krisztian Kasa)

Co-authored-by: Janos Kovacs <jkovacs@cloudera.com>
3 weeks agoHIVE-26184: COLLECT_SET with GROUP BY is very slow when some keys are highly skewed...
okumin [Mon, 13 Jun 2022 15:26:36 +0000 (00:26 +0900)] 
HIVE-26184: COLLECT_SET with GROUP BY is very slow when some keys are highly skewed (#3253) (okumin reviewed by Zoltan Haindrich)

3 weeks agoHIVE-26307: Avoid FS init in FileIO::newInputFile in vectorized Iceberg reads (Peter...
pvary [Mon, 13 Jun 2022 15:20:53 +0000 (17:20 +0200)] 
HIVE-26307: Avoid FS init in FileIO::newInputFile in vectorized Iceberg reads (Peter Vary reviewed by Adam Szita) (#3354)

3 weeks agoHIVE-25733: Add check-spelling/check-spelling (#2809) (Josh Soref reviewed by Zoltan...
Josh Soref [Mon, 13 Jun 2022 15:05:41 +0000 (11:05 -0400)] 
HIVE-25733: Add check-spelling/check-spelling (#2809) (Josh Soref reviewed by Zoltan Haindrich)

3 weeks agoHIVE-26165: Remove READ locks for ACID tables (Denys Kuzmenko, reviewed by Karen...
Denys Kuzmenko [Mon, 13 Jun 2022 07:14:09 +0000 (09:14 +0200)] 
HIVE-26165: Remove READ locks for ACID tables (Denys Kuzmenko, reviewed by Karen Coppage)

Closes #3235

3 weeks agoHIVE-26238: Decouple sort filter predicates optimization from digest normalization...
Stamatis Zampetakis [Tue, 17 May 2022 13:20:06 +0000 (15:20 +0200)] 
HIVE-26238: Decouple sort filter predicates optimization from digest normalization in CBO (Stamatis Zampetakis, reviewed by Zoltan Haindrich)

1. Decouple sort filter optimization from digest normalization by
refactoring HiveSortFilterPredicates into a (DFS) visitor. We cannot
use planner or rules cause they make use of digest. Performing this
optimization using a visitor slightly simplifies the code since there
is no need to have a registry since we are not going to visit the same
node twice.

2. Move the optimization after all post-join transformations to avoid
having other optimizations cancel the benefit of the sort filter
predicates.

Closes #3299

3 weeks agoHIVE-26301: Fix ACID tables bootstrap during reverse replication in unplanned failove...
Haymant Mangla [Fri, 10 Jun 2022 10:36:58 +0000 (16:06 +0530)] 
HIVE-26301: Fix ACID tables bootstrap during reverse replication in unplanned failover (Haymant Mangla reviewed by Peter Vary) (#3352)

3 weeks agoDisable flaky test
Peter Vary [Thu, 9 Jun 2022 16:10:19 +0000 (18:10 +0200)] 
Disable flaky test

3 weeks agoHIVE-26300: Upgraded Jackson bom version to 2.12.6.1+ to avoid CVE-2020-36518 (#3351...
Sai Hemanth Gantasala [Thu, 9 Jun 2022 10:16:56 +0000 (15:46 +0530)] 
HIVE-26300: Upgraded Jackson bom version to 2.12.6.1+ to avoid CVE-2020-36518 (#3351) (Sai Hemanth Gantasala reviewed by Zoltan Haindrich and Ayush Saxena)

3 weeks agoHIVE-26296: RuntimeException when executing EXPLAIN CBO JOINCOST on query with JDBC...
Stamatis Zampetakis [Tue, 7 Jun 2022 15:02:12 +0000 (17:02 +0200)] 
HIVE-26296: RuntimeException when executing EXPLAIN CBO JOINCOST on query with JDBC tables (Stamatis Zampetakis, reviewed by Alessandro Solimando, Krisztian Kasa)

Compute selectivity for all types of joins in the same way. There is no
particular reason to throw an exception when the Join operator is not
an instance of HiveJoin.

Closes #3349

3 weeks agoHIVE-26290: Remove useless calls to DateTimeFormatter#withZone without assignment...
Stamatis Zampetakis [Fri, 3 Jun 2022 16:46:47 +0000 (18:46 +0200)] 
HIVE-26290: Remove useless calls to DateTimeFormatter#withZone without assignment (Stamatis Zampetakis, reviewed by Ayush Saxena)

Closes #3342

3 weeks agoHIVE-26278: Add unit tests for Hive#getPartitionsByNames using batching (Stamatis...
Stamatis Zampetakis [Tue, 8 Feb 2022 15:56:56 +0000 (16:56 +0100)] 
HIVE-26278: Add unit tests for Hive#getPartitionsByNames using batching (Stamatis Zampetakis, reviewed by Zoltan Haindrich, Krisztian Kasa, Ayush Saxena)

Ensure that ValidWriteIdList is set when batching is involved in
getPartitionByNames.

Closes #3335

3 weeks agoHIVE-26293: Migrate remaining exclusive DDL operations to EXCL_WRITE lock & bug fixes...
Denys Kuzmenko [Thu, 9 Jun 2022 08:52:13 +0000 (10:52 +0200)] 
HIVE-26293: Migrate remaining exclusive DDL operations to EXCL_WRITE lock & bug fixes (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3103

3 weeks agoHIVE-26285: Overwrite database metadata on original source in optimised failover...
Haymant Mangla [Thu, 9 Jun 2022 07:41:05 +0000 (13:11 +0530)] 
HIVE-26285: Overwrite database metadata on original source in optimised failover. (Haymant Mangla reviewed by Denys Kuzmenko and Peter Vary) (#3346)

4 weeks agoHIVE-26268: Upgrade Snappy to 1.1.8.4 (#3326) (Sylwester Lachiewicz reviewed by Zolta...
Sylwester Lachiewicz [Wed, 8 Jun 2022 16:02:29 +0000 (18:02 +0200)] 
HIVE-26268: Upgrade Snappy to 1.1.8.4 (#3326) (Sylwester Lachiewicz reviewed by Zoltan Haindrich)

4 weeks agoHIVE-25635: Upgrade libthrift from 0.14.1 to 0.16.0 (#3327) (Sylwester Lachiewicz...
Sylwester Lachiewicz [Wed, 8 Jun 2022 16:00:49 +0000 (18:00 +0200)] 
HIVE-25635: Upgrade libthrift from 0.14.1 to 0.16.0 (#3327) (Sylwester Lachiewicz reviewed by Zoltan Haindrich)

4 weeks agoHIVE-21160: Rewrite Update statement as Multi-insert and do Update split early (Krisz...
Krisztian Kasa [Wed, 8 Jun 2022 14:19:47 +0000 (16:19 +0200)] 
HIVE-21160: Rewrite Update statement as Multi-insert and do Update split early (Krisztian Kasa, reviewed by Peter Vary, Zoltan Haindrich)

4 weeks agoHIVE-26266: Column information is missing from lineage for CTAS when custom/translate...
Sourabh Badhya [Wed, 8 Jun 2022 07:59:20 +0000 (13:29 +0530)] 
HIVE-26266: Column information is missing from lineage for CTAS when custom/translated location is used (Sourabh Badhya, reviewed by  Zoltan Haindrich)

Closes #3331

4 weeks agoHIVE-26289: Remove useless try catch in DataWritableReadSupport#getWriterDateProlepti...
Stamatis Zampetakis [Fri, 3 Jun 2022 16:14:22 +0000 (18:14 +0200)] 
HIVE-26289: Remove useless try catch in DataWritableReadSupport#getWriterDateProleptic (Stamatis Zampetakis, reviewed by Ayush Saxena)

Closes #3341

4 weeks agoHIVE-26217: Make CTAS use Direct Insert Semantics (Sourabh Badhya, reviewed by Denys... 3148/head
Sourabh Badhya [Sat, 4 Jun 2022 07:04:42 +0000 (12:34 +0530)] 
HIVE-26217: Make CTAS use Direct Insert Semantics (Sourabh Badhya, reviewed by Denys Kuzmenko, Peter Vary, Sai Hemanth Gantasala)

Closes #3281

4 weeks agoHIVE-26267: Addendum to HIVE-26107: Issue with the PreparedStatement in TxnHandler...
veghlaci05 [Fri, 3 Jun 2022 12:00:36 +0000 (14:00 +0200)] 
HIVE-26267: Addendum to HIVE-26107: Issue with the PreparedStatement in TxnHandler#compact (Laszlo Vegh, reviewed by Denys Kuzmenko)

Closes #3325

4 weeks agoHIVE-26282: Improve iceberg CTAS error message for unsupported types (#3337) (Laszlo...
László Pintér [Fri, 3 Jun 2022 11:43:10 +0000 (13:43 +0200)] 
HIVE-26282: Improve iceberg CTAS error message for unsupported types (#3337) (Laszlo Pinter, reviewed by Peter Vary)

4 weeks agoHIVE-26270: Wrong timestamps when reading Hive 3.1.x Parquet files with vectorized...
Stamatis Zampetakis [Mon, 30 May 2022 10:18:06 +0000 (12:18 +0200)] 
HIVE-26270: Wrong timestamps when reading Hive 3.1.x Parquet files with vectorized reader (Stamatis Zampetakis, reviewed by Peter Vary)

Closes #3338

4 weeks agoHIVE-26279: Drop unused requests from TestHiveMetaStoreClientApiArgumentsChecker...
Stamatis Zampetakis [Tue, 8 Feb 2022 14:50:28 +0000 (15:50 +0100)] 
HIVE-26279: Drop unused requests from TestHiveMetaStoreClientApiArgumentsChecker (Stamatis Zampetakis, reviewed by Ayush Saxena)

Closes #3336

4 weeks agoHIVE-26228: Implement Iceberg table rollback feature (#3287) (Laszlo Pinter, reviewed...
László Pintér [Fri, 3 Jun 2022 06:26:58 +0000 (08:26 +0200)] 
HIVE-26228: Implement Iceberg table rollback feature (#3287) (Laszlo Pinter, reviewed by Adam Szita and Peter Vary)

4 weeks agoHIVE-26264: Iceberg integration: Fetch virtual columns on demand (Krisztian Kasa...
Krisztian Kasa [Fri, 3 Jun 2022 04:13:27 +0000 (06:13 +0200)] 
HIVE-26264: Iceberg integration: Fetch virtual columns on demand (Krisztian Kasa, reviewed by Peter Vary)

4 weeks agoHIVE-25421: Fallback from vectorization when reading Iceberg's time columns from...
Adam Szita [Thu, 2 Jun 2022 07:50:40 +0000 (09:50 +0200)] 
HIVE-25421: Fallback from vectorization when reading Iceberg's time columns from ORC files (#3334) (Adam Szita, reviewed by Laszlo Pinter)

4 weeks agoHIVE-26274: No vectorization if query has upper case window function (Krisztian Kasa...
Krisztian Kasa [Thu, 2 Jun 2022 06:54:39 +0000 (08:54 +0200)] 
HIVE-26274: No vectorization if query has upper case window function (Krisztian Kasa, reviewed by Laszlo Bodor)

4 weeks agoHIVE-26230: Option to URL encode special chars in hbase.column.mapping that are valid...
Adam Szita [Wed, 1 Jun 2022 20:41:53 +0000 (22:41 +0200)] 
HIVE-26230: Option to URL encode special chars in hbase.column.mapping that are valid HBase column family chars (#3314) (Adam Szita, reviewed by Peter Vary)

5 weeks agoHIVE-26272: Inline util code that is used from log4j jar (#3330). (PJ Fanning, review...
PJ Fanning [Wed, 1 Jun 2022 17:48:04 +0000 (18:48 +0100)] 
HIVE-26272: Inline util code that is used from log4j jar (#3330). (PJ Fanning, reviewed by Ayush Saxena)

5 weeks agoHIVE-25907: IOW Directory queries fails to write data to final path when query result...
Syed Shameerur Rahman [Wed, 1 Jun 2022 09:18:37 +0000 (14:48 +0530)] 
HIVE-25907: IOW Directory queries fails to write data to final path when query result cache is enabled (#2978) (Syed Shameerur Rahman reviewed by Peter Vary)

5 weeks agoHIVE-26224: Add support for ESRI GeoSpatial SERDE formats. (#3300). (Ayush Saxena...
Ayush Saxena [Tue, 31 May 2022 08:48:05 +0000 (14:18 +0530)] 
HIVE-26224: Add support for ESRI GeoSpatial SERDE formats. (#3300). (Ayush Saxena, reviewed by Mahesh Kumar Behera)

5 weeks agoHIVE-22670: ArrayIndexOutOfBoundsException when vectorized reader is (#3328) (Ganesha...
achennagiri [Tue, 31 May 2022 07:05:49 +0000 (00:05 -0700)] 
HIVE-22670: ArrayIndexOutOfBoundsException when vectorized reader is (#3328) (Ganesha Shreedhara and Abhay Chennagiri reviewed by Peter Vary)

5 weeks agoHIVE-26239 Shutdown Hash table load executor service threads when they are interrupte...
Ramesh Kumar [Fri, 27 May 2022 15:48:19 +0000 (08:48 -0700)] 
HIVE-26239 Shutdown Hash table load executor service threads when they are interrupted (#3302)

5 weeks agoHIVE-26260: Use `Reader.getSchema` instead of deprecated `Reader.getTypes` (#3318)
Dongjoon Hyun [Thu, 26 May 2022 22:23:13 +0000 (15:23 -0700)] 
HIVE-26260: Use `Reader.getSchema` instead of deprecated `Reader.getTypes` (#3318)

### What changes were proposed in this pull request?

This PR aims to use `org.apache.orc.Reader.getSchema` instead of the deprecated `org.apache.orc.Reader.getTypes` API.

### Why are the changes needed?

`getTypes` was deprecated.

- https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/Reader.java#L144
```java
  /**
   * Get the list of types contained in the file. The root type is the first
   * type in the list.
   * @return the list of flattened types
   * @deprecated use getSchema instead
   * @since 1.1.0
   */
  List<OrcProto.Type> getTypes();
```

In addition, AS-IS implementation is only a slow-wrapper.
- https://github.com/apache/orc/blob/1e2962064b209f1b00188877f08d4226da85c640/java/core/src/java/org/apache/orc/impl/ReaderImpl.java#L259-L262
```java
  @Override
  public List<OrcProto.Type> getTypes() {
    return OrcUtils.getOrcTypes(schema);
  }
```

- https://github.com/apache/orc/blob/1e2962064b209f1b00188877f08d4226da85c640/java/core/src/java/org/apache/orc/OrcUtils.java#L108-L112
```java
  public static List<OrcProto.Type> getOrcTypes(TypeDescription typeDescr) {
    List<OrcProto.Type> result = new ArrayList<>();
    appendOrcTypes(result, typeDescr);
    return result;
  }
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs

5 weeks agoHIVE-26233: Problems reading back PARQUET timestamps above 10000 years (#3295) (Peter...
Peter Vary [Thu, 26 May 2022 15:22:15 +0000 (17:22 +0200)] 
HIVE-26233: Problems reading back PARQUET timestamps above 10000 years (#3295) (Peter Vary reviewed by  Stamatis Zampetakis) (Addendum)

5 weeks agoHIVE-26233: Problems reading back PARQUET timestamps above 10000 years (#3295) (Peter...
pvary [Thu, 26 May 2022 15:15:35 +0000 (17:15 +0200)] 
HIVE-26233: Problems reading back PARQUET timestamps above 10000 years (#3295) (Peter Vary reviewed by  Stamatis Zampetakis)

5 weeks agoHIVE-24598: handle null in trim function (Shoubhik Chakraborty, reviewed by Krisztian...
Soubhik Chakraborty [Thu, 26 May 2022 12:50:14 +0000 (08:50 -0400)] 
HIVE-24598: handle null in trim function (Shoubhik Chakraborty, reviewed by Krisztian Kasa)

Co-authored-by: Soubhik Chakraborty <shoubhik.chakraborty@cloudera.com>
5 weeks agoHIVE-25589: SQL: Implement HAVING/QUALIFY predicates for ROW_NUMBER()=1 (Krisztian...
Krisztian Kasa [Thu, 26 May 2022 11:22:46 +0000 (13:22 +0200)] 
HIVE-25589: SQL: Implement HAVING/QUALIFY predicates for ROW_NUMBER()=1 (Krisztian Kasa, reviewed by Zoltan Haindrich)

5 weeks agoHIVE-26210: Fix tests for Cleaner failed attempt threshold (Laszlo Vegh, reviewed...
veghlaci05 [Thu, 26 May 2022 09:30:50 +0000 (11:30 +0200)] 
HIVE-26210: Fix tests for Cleaner failed attempt threshold (Laszlo Vegh, reviewed by Denys Kuzmenko)

Closes #3274

5 weeks agoHIVE-26261: Fix some issues with Spark engine removal (#3320) (Peter Vary reviewed...
pvary [Thu, 26 May 2022 08:43:01 +0000 (10:43 +0200)] 
HIVE-26261: Fix some issues with Spark engine removal (#3320) (Peter Vary reviewed by Zoltan Haindrich)

5 weeks agoHIVE-26235: OR Condition on binary column is returning empty result (#3305) (Peter...
pvary [Thu, 26 May 2022 04:42:34 +0000 (06:42 +0200)] 
HIVE-26235: OR Condition on binary column is returning empty result (#3305) (Peter Vary, reviewed by Laszlo Bodor)

6 weeks agodisable flaky mysql metastore test
Zoltan Haindrich [Wed, 25 May 2022 11:41:41 +0000 (11:41 +0000)] 
disable flaky mysql metastore test

6 weeks agoDisable flaky tests
Peter Vary [Tue, 24 May 2022 13:41:42 +0000 (15:41 +0200)] 
Disable flaky tests

6 weeks agoHIVE-26258: Provide an option for enable locking of external tables (#3313) (Peter...
pvary [Tue, 24 May 2022 11:58:15 +0000 (13:58 +0200)] 
HIVE-26258: Provide an option for enable locking of external tables (#3313) (Peter Vary reviewed by Denys Kuzmenko)

6 weeks agoHIVE-26252: Missing locks in case of MERGE with multiple branches (Denys Kuzmenko...
Denys Kuzmenko [Tue, 24 May 2022 09:45:25 +0000 (11:45 +0200)] 
HIVE-26252: Missing locks in case of MERGE with multiple branches (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3308

6 weeks agoHIVE-26251: Disable vectorization when querying iceberg metadata tables (#3306) ...
László Pintér [Mon, 23 May 2022 13:11:00 +0000 (15:11 +0200)] 
HIVE-26251: Disable vectorization when querying iceberg metadata tables (#3306) (Laszlo Pinter, reviewed by Adam Szita)

6 weeks agoHIVE-26237: Check if replication cause metastore connection leakage (Haymant Mangla...
Haymant Mangla [Mon, 23 May 2022 06:56:37 +0000 (12:26 +0530)] 
HIVE-26237: Check if replication cause metastore connection leakage (Haymant Mangla, reviewed by Denys Kuzmenko, Peter Vary)

Closes #3298

6 weeks agoHIVE-26223: Integrate ESRI GeoSpatial UDFs. (#3283). (Ayush Saxena, reviewed by Mahes...
Ayush Saxena [Thu, 19 May 2022 04:16:35 +0000 (09:46 +0530)] 
HIVE-26223: Integrate ESRI GeoSpatial UDFs. (#3283). (Ayush Saxena, reviewed by Mahesh Kumar Behera)

7 weeks agoHIVE-26231: Generate insert notification events when dynamic partition insert is...
SourabhBadhya [Wed, 18 May 2022 07:59:49 +0000 (13:29 +0530)] 
HIVE-26231: Generate insert notification events when dynamic partition insert is done on existing partitions ( Sourabh Badhya, reviewed by Denys Kuzmenko)

Closes #3294

7 weeks agoHIVE-26225: Delete operations in ObjectStore.cleanWriteNotificationEvents should...
Haymant Mangla [Wed, 18 May 2022 01:48:00 +0000 (07:18 +0530)] 
HIVE-26225: Delete operations in ObjectStore.cleanWriteNotificationEvents should be performed in different transactions (Haymant Mangla, reviewed by Mahesh Kumar Behera)

7 weeks agoHIVE-25444: Make tables based on storage handlers authorization (HIVE-24705) configur...
Adam Szita [Tue, 17 May 2022 13:19:05 +0000 (15:19 +0200)] 
HIVE-25444: Make tables based on storage handlers authorization (HIVE-24705) configurable (#3290) (originally contributed by Sai Hemanth Gantasala, committed by Adam Szita, reviewed by Peter Vary)

7 weeks agoHIVE-26226: Exclude jdk.tools dep from hive-metastore in upgrade-acid (Sylwester...
Sylwester Lachiewicz [Thu, 12 May 2022 09:58:00 +0000 (11:58 +0200)] 
HIVE-26226: Exclude jdk.tools dep from hive-metastore in upgrade-acid (Sylwester Lachiewicz, reviewed by Stamatis Zampetakis)

The jdk.tools jars are not present in java versions > 8 thus there are
build problems when compiling with newer JDKs.

Exclude the dependency from hive-metastore (2.3.3) to avoid compilation
problems in recent JDKs.

It is safe to so because the dependency will still be fetched
transitively by hadoop-common (2.7.2) when the appropriate maven (JDK)
profile is in use.

Closes #3284

7 weeks agoHIVE-26173: Upgrade derby to 10.14.2.0 (Hemanth Boyina, reviewed by Stamatis Zampetakis)
hemanthboyina [Mon, 25 Apr 2022 17:20:12 +0000 (22:50 +0530)] 
HIVE-26173: Upgrade derby to 10.14.2.0 (Hemanth Boyina, reviewed by Stamatis Zampetakis)

Closes #3243

7 weeks agoHIVE-25872: Skip tracking of alterDatabase events for replication specific properties...
Haymant Mangla [Tue, 17 May 2022 05:02:34 +0000 (10:32 +0530)] 
HIVE-25872: Skip tracking of alterDatabase events for replication specific properties (#2950).  (Haymant Mangla, reviewed by Ayush Saxena andRajesh Balamohan)

7 weeks agoHIVE-26205: Incorrect scope for slf4j-api dependency in kafka-handler (Wechar Yu...
wecharyu [Sat, 7 May 2022 12:56:54 +0000 (20:56 +0800)] 
HIVE-26205: Incorrect scope for slf4j-api dependency in kafka-handler (Wechar Yu, reviewed by Stamatis Zampetakis)

The classes in the kafka-handler module are using the slf4j-api thus
this dependency must be set at compile scope. Currently it is set at
test scope which makes the build fail in some recent maven versions (
e.g., 3.8.5).

The parent pom declares explicitly slf4j-api at compile scope so
removing all the references from kafka-handler/pom.xml is the way to
go.

Closes #3272

7 weeks agoHIVE-26157: Change Iceberg storage handler authz URI to metadata location. (#3226...
László Pintér [Fri, 13 May 2022 06:44:11 +0000 (08:44 +0200)] 
HIVE-26157: Change Iceberg storage handler authz URI to metadata location. (#3226) (Laszlo Pinter, reviewed by Peter Vary)