hive.git
39 min agoHIVE-25589: SQL: Implement HAVING/QUALIFY predicates for ROW_NUMBER()=1 (Krisztian... master
Krisztian Kasa [Thu, 26 May 2022 11:22:46 +0000 (13:22 +0200)] 
HIVE-25589: SQL: Implement HAVING/QUALIFY predicates for ROW_NUMBER()=1 (Krisztian Kasa, reviewed by Zoltan Haindrich)

2 hours agoHIVE-26210: Fix tests for Cleaner failed attempt threshold (Laszlo Vegh, reviewed...
veghlaci05 [Thu, 26 May 2022 09:30:50 +0000 (11:30 +0200)] 
HIVE-26210: Fix tests for Cleaner failed attempt threshold (Laszlo Vegh, reviewed by Denys Kuzmenko)

Closes #3274

3 hours agoHIVE-26261: Fix some issues with Spark engine removal (#3320) (Peter Vary reviewed...
pvary [Thu, 26 May 2022 08:43:01 +0000 (10:43 +0200)] 
HIVE-26261: Fix some issues with Spark engine removal (#3320) (Peter Vary reviewed by Zoltan Haindrich)

7 hours agoHIVE-26235: OR Condition on binary column is returning empty result (#3305) (Peter...
pvary [Thu, 26 May 2022 04:42:34 +0000 (06:42 +0200)] 
HIVE-26235: OR Condition on binary column is returning empty result (#3305) (Peter Vary, reviewed by Laszlo Bodor)

24 hours agodisable flaky mysql metastore test
Zoltan Haindrich [Wed, 25 May 2022 11:41:41 +0000 (11:41 +0000)] 
disable flaky mysql metastore test

46 hours agoDisable flaky tests
Peter Vary [Tue, 24 May 2022 13:41:42 +0000 (15:41 +0200)] 
Disable flaky tests

2 days agoHIVE-26258: Provide an option for enable locking of external tables (#3313) (Peter...
pvary [Tue, 24 May 2022 11:58:15 +0000 (13:58 +0200)] 
HIVE-26258: Provide an option for enable locking of external tables (#3313) (Peter Vary reviewed by Denys Kuzmenko)

2 days agoHIVE-26252: Missing locks in case of MERGE with multiple branches (Denys Kuzmenko...
Denys Kuzmenko [Tue, 24 May 2022 09:45:25 +0000 (11:45 +0200)] 
HIVE-26252: Missing locks in case of MERGE with multiple branches (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3308

2 days agoHIVE-26251: Disable vectorization when querying iceberg metadata tables (#3306) ...
László Pintér [Mon, 23 May 2022 13:11:00 +0000 (15:11 +0200)] 
HIVE-26251: Disable vectorization when querying iceberg metadata tables (#3306) (Laszlo Pinter, reviewed by Adam Szita)

3 days agoHIVE-26237: Check if replication cause metastore connection leakage (Haymant Mangla...
Haymant Mangla [Mon, 23 May 2022 06:56:37 +0000 (12:26 +0530)] 
HIVE-26237: Check if replication cause metastore connection leakage (Haymant Mangla, reviewed by Denys Kuzmenko, Peter Vary)

Closes #3298

7 days agoHIVE-26223: Integrate ESRI GeoSpatial UDFs. (#3283). (Ayush Saxena, reviewed by Mahes...
Ayush Saxena [Thu, 19 May 2022 04:16:35 +0000 (09:46 +0530)] 
HIVE-26223: Integrate ESRI GeoSpatial UDFs. (#3283). (Ayush Saxena, reviewed by Mahesh Kumar Behera)

8 days agoHIVE-26231: Generate insert notification events when dynamic partition insert is...
SourabhBadhya [Wed, 18 May 2022 07:59:49 +0000 (13:29 +0530)] 
HIVE-26231: Generate insert notification events when dynamic partition insert is done on existing partitions ( Sourabh Badhya, reviewed by Denys Kuzmenko)

Closes #3294

8 days agoHIVE-26225: Delete operations in ObjectStore.cleanWriteNotificationEvents should...
Haymant Mangla [Wed, 18 May 2022 01:48:00 +0000 (07:18 +0530)] 
HIVE-26225: Delete operations in ObjectStore.cleanWriteNotificationEvents should be performed in different transactions (Haymant Mangla, reviewed by Mahesh Kumar Behera)

8 days agoHIVE-25444: Make tables based on storage handlers authorization (HIVE-24705) configur...
Adam Szita [Tue, 17 May 2022 13:19:05 +0000 (15:19 +0200)] 
HIVE-25444: Make tables based on storage handlers authorization (HIVE-24705) configurable (#3290) (originally contributed by Sai Hemanth Gantasala, committed by Adam Szita, reviewed by Peter Vary)

9 days agoHIVE-26226: Exclude jdk.tools dep from hive-metastore in upgrade-acid (Sylwester...
Sylwester Lachiewicz [Thu, 12 May 2022 09:58:00 +0000 (11:58 +0200)] 
HIVE-26226: Exclude jdk.tools dep from hive-metastore in upgrade-acid (Sylwester Lachiewicz, reviewed by Stamatis Zampetakis)

The jdk.tools jars are not present in java versions > 8 thus there are
build problems when compiling with newer JDKs.

Exclude the dependency from hive-metastore (2.3.3) to avoid compilation
problems in recent JDKs.

It is safe to so because the dependency will still be fetched
transitively by hadoop-common (2.7.2) when the appropriate maven (JDK)
profile is in use.

Closes #3284

9 days agoHIVE-26173: Upgrade derby to 10.14.2.0 (Hemanth Boyina, reviewed by Stamatis Zampetakis)
hemanthboyina [Mon, 25 Apr 2022 17:20:12 +0000 (22:50 +0530)] 
HIVE-26173: Upgrade derby to 10.14.2.0 (Hemanth Boyina, reviewed by Stamatis Zampetakis)

Closes #3243

9 days agoHIVE-25872: Skip tracking of alterDatabase events for replication specific properties...
Haymant Mangla [Tue, 17 May 2022 05:02:34 +0000 (10:32 +0530)] 
HIVE-25872: Skip tracking of alterDatabase events for replication specific properties (#2950).  (Haymant Mangla, reviewed by Ayush Saxena andRajesh Balamohan)

9 days agoHIVE-26205: Incorrect scope for slf4j-api dependency in kafka-handler (Wechar Yu...
wecharyu [Sat, 7 May 2022 12:56:54 +0000 (20:56 +0800)] 
HIVE-26205: Incorrect scope for slf4j-api dependency in kafka-handler (Wechar Yu, reviewed by Stamatis Zampetakis)

The classes in the kafka-handler module are using the slf4j-api thus
this dependency must be set at compile scope. Currently it is set at
test scope which makes the build fail in some recent maven versions (
e.g., 3.8.5).

The parent pom declares explicitly slf4j-api at compile scope so
removing all the references from kafka-handler/pom.xml is the way to
go.

Closes #3272

13 days agoHIVE-26157: Change Iceberg storage handler authz URI to metadata location. (#3226...
László Pintér [Fri, 13 May 2022 06:44:11 +0000 (08:44 +0200)] 
HIVE-26157: Change Iceberg storage handler authz URI to metadata location. (#3226) (Laszlo Pinter, reviewed by Peter Vary)

13 days agoHIVE-25230: Add position and occurrence to INSTR() (#2378) (Quanlong Huang, reviewed...
Quanlong Huang [Fri, 13 May 2022 01:14:58 +0000 (09:14 +0800)] 
HIVE-25230: Add position and occurrence to INSTR() (#2378) (Quanlong Huang, reviewed by Zhihua Deng)

13 days agoHIVE-26071: JWT authentication mechanism for Thrift over HTTP in HiveMetastore (...
Sourabh Goyal [Thu, 12 May 2022 18:40:56 +0000 (11:40 -0700)] 
HIVE-26071: JWT authentication mechanism for Thrift over HTTP in HiveMetastore (#3233) (Sourabh Goyal, reviewed by Yu-wen, Deng and Sai)

What changes were proposed in this pull request?
This PR is a follow up of #3105. It adds a support for JWT authentication in HiveMetastore server when run in HTTP transport mode.

Why are the changes needed?
It supports a new authentication mechanism ie JWT in HiveMetastore server.

Does this PR introduce any user-facing change?
No

How was this patch tested?
Added new unit tests that cover cases like

successfully authenticating valid JWT
failing to authenticate expired, invalid JWTs

* Add JWTValidator and URLBasedJWKSProvider code from HS2

Change-Id: I969f57daf640adb16f228e95b1b522f8ffc24ffe

* Add JWT authentication in HiveMetastore

Change-Id: I6d84517a1ee97df492ad3816ec866c0b785ed5ed

* Better error handling for authentication failures. Added integration tests for validating JWT

Change-Id: I6b9da531db4e4a805d8daa1ba6d941c5643bf514

* Added test JWTs for jwt authentication tests

Change-Id: Ice36a703d8af7d4dbf28a48c9bb96127100fd8c7

* moved jwt test keys under jwt directory

Change-Id: I8bf0b4bbc101a0acb3f69bb1963b9c4bcda5b719

* Fixes failures in metastore jwt unit tests

Change-Id: I2877730a34dff7d3184b100ec04031032611838a

* Addresses review comments

Change-Id: I8498e85212476c663cf735211848a28baaa3bad5

* Addresses nits from review comments

Change-Id: Id67588c106104732a0f6e49e5c983cb5f7287c3e

* Added more comments in the code

Change-Id: Ia51f490362985d109778a6a0aa92a281436d5d21

* removes unsed import statement

Change-Id: I94633bdce0db87a9085968dde79d8ff6cd9bf4a3

13 days agoHIVE-26202: Refactor Iceberg Writers (Peter Vary reviewed by Laszlo Pinter) (#3269)
pvary [Thu, 12 May 2022 13:53:16 +0000 (15:53 +0200)] 
HIVE-26202: Refactor Iceberg Writers (Peter Vary reviewed by Laszlo Pinter) (#3269)

2 weeks agoHIVE-26172: Upgrade ant to 1.10.12 (Hemanth Boyina, reviewed by Stamatis Zampetakis)
hemanthboyina [Mon, 25 Apr 2022 16:55:29 +0000 (22:25 +0530)] 
HIVE-26172: Upgrade ant to 1.10.12 (Hemanth Boyina, reviewed by Stamatis Zampetakis)

Closes #3242

2 weeks agoHIVE-26158: TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after...
Zoltan Haindrich [Wed, 11 May 2022 15:10:51 +0000 (17:10 +0200)] 
HIVE-26158: TRANSLATED_TO_EXTERNAL partition tables cannot query partition data after rename table (#3255) (Zoltan Haindrich reviewed by Saihemanth Gantasala)

2 weeks agoHIVE-26203: Implement alter iceberg table metadata location (#3270) (Laszlo Pinter...
László Pintér [Wed, 11 May 2022 11:57:08 +0000 (13:57 +0200)] 
HIVE-26203: Implement alter iceberg table metadata location (#3270) (Laszlo Pinter, reviewed by Peter Vary)

2 weeks agoHIVE-26215: Expose the MIN_HISTORY_LEVEL table through Hive sys database (#3275)...
Simhadri Govindappa [Wed, 11 May 2022 11:43:16 +0000 (17:13 +0530)] 
HIVE-26215: Expose the MIN_HISTORY_LEVEL table through Hive sys database (#3275). (Simhadri G, reviewed by Ayush Saxena)

2 weeks agoHIVE-26219 Encapsulate the API change for FileUtils.isActionPermittedForFileHierarchy...
Ramesh Kumar [Tue, 10 May 2022 23:17:25 +0000 (16:17 -0700)] 
HIVE-26219 Encapsulate the API change for FileUtils.isActionPermittedForFileHierarchy (#3278)

2 weeks agoHVIE-26199 Reduce FileSystem init during user impersonation (#3264)
Ramesh Kumar [Tue, 10 May 2022 14:44:50 +0000 (07:44 -0700)] 
HVIE-26199 Reduce FileSystem init during user impersonation (#3264)

2 weeks agoHIVE-25969: Unable to reference table column named default (Krisztian Kasa, reviewed...
Krisztian Kasa [Tue, 10 May 2022 11:46:13 +0000 (13:46 +0200)] 
HIVE-25969: Unable to reference table column named default (Krisztian Kasa, reviewed by Zoltan Haindrich)

2 weeks agoHIVE-26177: Create a new connection pool for compaction (DataNucleus) (Antal Sinkovit... 3224/head
Antal Sinkovits [Mon, 9 May 2022 07:49:45 +0000 (09:49 +0200)] 
HIVE-26177: Create a new connection pool for compaction (DataNucleus) (Antal Sinkovits, reviewed by Denys Kuzmenko)

Closes #3265

2 weeks agoHIVE-26136: Implement UPDATE statements for Iceberg tables (Peter Vary reviewed by...
pvary [Mon, 9 May 2022 05:36:12 +0000 (07:36 +0200)] 
HIVE-26136: Implement UPDATE statements for Iceberg tables (Peter Vary reviewed by Laszlo Pinter) (#3204)

2 weeks agoHIVE-26190: Implement create iceberg table with metadata location. (#3259) (Laszlo...
László Pintér [Fri, 6 May 2022 13:13:07 +0000 (15:13 +0200)] 
HIVE-26190: Implement create iceberg table with metadata location. (#3259) (Laszlo Pinter, reviewed by Peter Vary)

2 weeks agoHIVE-26200: Add tests for Iceberg DELETE statements for every supported type (Peter...
pvary [Fri, 6 May 2022 06:47:47 +0000 (08:47 +0200)] 
HIVE-26200: Add tests for Iceberg DELETE statements for every supported type (Peter Vary reviewed by Laszlo Pinter) (#3268)

2 weeks agoHIVE-26023: Non blocking REPLACE, RENAME COLUMNS (Denys Kuzmenko, reviewed by Antal...
Denys Kuzmenko [Thu, 5 May 2022 18:20:16 +0000 (20:20 +0200)] 
HIVE-26023: Non blocking REPLACE, RENAME COLUMNS (Denys Kuzmenko, reviewed by Antal Sinkovits, Laszlo Vegh)

Closes #3089

2 weeks agoHIVE-23391: Non blocking ALTER TABLE add column (Denys Kuzmenko, reviewed by Antal...
Denys Kuzmenko [Thu, 5 May 2022 13:22:44 +0000 (15:22 +0200)] 
HIVE-23391: Non blocking ALTER TABLE add column (Denys Kuzmenko, reviewed by Antal Sinkovits, Laszlo Vegh)

Closes #3083

3 weeks agoHIVE-26183: Create delete writer for the UPDATE statements (Peter Vary reviewed by...
pvary [Wed, 4 May 2022 08:50:04 +0000 (10:50 +0200)] 
HIVE-26183: Create delete writer for the UPDATE statements (Peter Vary reviewed by Adam Szita and Marton Bod) (#3251)

3 weeks agoHIVE-26076: Non blocking ADD PARTITION if not exists (Denys Kuzmenko, reviewed by...
Denys Kuzmenko [Wed, 4 May 2022 07:51:33 +0000 (09:51 +0200)] 
HIVE-26076: Non blocking ADD PARTITION if not exists (Denys Kuzmenko, reviewed by Antal Sinkovits, Laszlo Vegh)

Closes #3122

3 weeks agoHIVE-26106: Non blocking ADD, DROP constraint (Denys Kuzmenko, reviewed by Laszlo...
Denys Kuzmenko [Tue, 3 May 2022 12:30:23 +0000 (14:30 +0200)] 
HIVE-26106: Non blocking ADD, DROP constraint (Denys Kuzmenko, reviewed by Laszlo Vegh)

Closes #3165

3 weeks agoHIVE-26189: Iceberg metadata query throws exceptions after partition evolution (adden...
Adam Szita [Tue, 3 May 2022 06:59:46 +0000 (08:59 +0200)] 
HIVE-26189: Iceberg metadata query throws exceptions after partition evolution (addendum) (#3262) (Adam Szita)

3 weeks agoHIVE-26189: Iceberg metadata query throws exceptions after partition evolution (...
Adam Szita [Mon, 2 May 2022 09:22:21 +0000 (11:22 +0200)] 
HIVE-26189: Iceberg metadata query throws exceptions after partition evolution (#3258) (Adam Szita, reviewed by Peter Vary)

3 weeks agoHIVE-26107: Worker shouldn't inject duplicate entries in `ready for cleaning` state...
veghlaci05 [Mon, 2 May 2022 09:20:36 +0000 (11:20 +0200)] 
HIVE-26107: Worker shouldn't inject duplicate entries in `ready for cleaning` state into the compaction queue (Laszlo Vegh, reviewed by Denys Kuzmenko, Karen Coppage)

Closes #3172

3 weeks agoHIVE-26193: Fix Iceberg partitioned tables null bucket handling (Peter Vary reviewed...
pvary [Mon, 2 May 2022 06:35:29 +0000 (08:35 +0200)] 
HIVE-26193: Fix Iceberg partitioned tables null bucket handling (Peter Vary reviewed by Marton Bod and Adam Szita) (#3261)

3 weeks agoHIVE-26167: QueryStateMap is SessionState is not maintained correctly. (#3234) (Laszl...
László Pintér [Sat, 30 Apr 2022 15:05:06 +0000 (17:05 +0200)] 
HIVE-26167: QueryStateMap is SessionState is not maintained correctly. (#3234) (Laszlo Pinter, reviewed by Peter Vary)

3 weeks agoHIVE-26149: Non blocking DROP DATABASE implementation (Denys Kuzmenko, reviewed by...
Denys Kuzmenko [Fri, 29 Apr 2022 06:38:53 +0000 (08:38 +0200)] 
HIVE-26149: Non blocking DROP DATABASE implementation (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3220

4 weeks agoHIVE-26176: Create a new connection pool for compaction (CompactionTxnHandler) (Antal...
Antal Sinkovits [Thu, 28 Apr 2022 10:33:24 +0000 (12:33 +0200)] 
HIVE-26176: Create a new connection pool for compaction (CompactionTxnHandler) (Antal Sinkovits, reviewed by Denys Kuzmenko and Peter Vary)

Closes #3223

4 weeks agoHIVE-25758: OOM due to recursive application of CBO rules (javadoc fix) (Alessandro...
Alessandro Solimando [Thu, 28 Apr 2022 08:01:37 +0000 (10:01 +0200)] 
HIVE-25758: OOM due to recursive application of CBO rules (javadoc fix) (Alessandro Solimando reviewed by Peter Vary) (#3252)

4 weeks agoHIVE-26160: Materialized View rewrite does not check tables scanned in sub-query...
Krisztian Kasa [Thu, 28 Apr 2022 04:10:11 +0000 (06:10 +0200)] 
HIVE-26160: Materialized View rewrite does not check tables scanned in sub-query expressions (Krisztian Kasa, reviewed by Stamatis Zampetakis)

4 weeks agoHIVE-26180: Change MySQLConnectorProvider driver from mariadb to mysql (#3250)
Butao Zhang [Thu, 28 Apr 2022 04:05:32 +0000 (12:05 +0800)] 
HIVE-26180: Change MySQLConnectorProvider driver from mariadb to mysql (#3250)

4 weeks agoHIVE-25091: Implemented connector provider for MSSQL and Oracle (#3167) (Saihemanth...
Sai Hemanth Gantasala [Thu, 28 Apr 2022 03:43:43 +0000 (20:43 -0700)] 
HIVE-25091: Implemented connector provider for MSSQL and Oracle (#3167) (Saihemanth Gantasale via Naveen Gangam)

4 weeks agoHIVE-26131: Incorrect OutputFormat when describing jdbc connector table (#3200)
Butao Zhang [Thu, 28 Apr 2022 03:32:32 +0000 (11:32 +0800)] 
HIVE-26131: Incorrect OutputFormat when describing jdbc connector table (#3200)

4 weeks agoHIVE-26032: Upgrade cron-utils to 9.1.6 (#3099)
Yuming Wang [Wed, 27 Apr 2022 19:28:29 +0000 (03:28 +0800)] 
HIVE-26032: Upgrade cron-utils to 9.1.6 (#3099)

4 weeks agoHIVE-26178: Exclude wood-stox jar in the packaging as it conflicts with hadoop's...
Sai Hemanth Gantasala [Wed, 27 Apr 2022 18:12:19 +0000 (11:12 -0700)] 
HIVE-26178: Exclude wood-stox jar in the packaging as it conflicts with hadoop's jar in spark class path (#3248) (Saihemanth Gantasala via Naveen Gangam)

4 weeks agoHIVE-25758: OOM due to recursive application of CBO rules (Alessandro Solimando,...
Alessandro Solimando [Mon, 24 Jan 2022 12:08:56 +0000 (13:08 +0100)] 
HIVE-25758: OOM due to recursive application of CBO rules (Alessandro Solimando, reviewed by Stamatis Zampetakis)

Closes #2966

4 weeks agoHIVE-26114: Fix jdbc connection hivesrerver2 using dfs command with prefix space...
shezm [Wed, 27 Apr 2022 06:44:58 +0000 (14:44 +0800)] 
HIVE-26114: Fix jdbc connection hivesrerver2 using dfs command with prefix space will cause exception (Shezm reviewed b Peter Vary) (#3176)

4 weeks agoHIVE-26170: Code cleanup in jdbc dataconnector (#3237)
Butao Zhang [Tue, 26 Apr 2022 16:08:34 +0000 (00:08 +0800)] 
HIVE-26170: Code cleanup in jdbc dataconnector (#3237)

4 weeks agoHIVE-26135: Invalid Anti join conversion may cause missing results (#3205) (Zoltan...
Zoltan Haindrich [Tue, 26 Apr 2022 15:22:49 +0000 (17:22 +0200)] 
HIVE-26135: Invalid Anti join conversion may cause missing results (#3205) (Zoltan Haindrich reviewed by Krisztian Kasa)

4 weeks agoHIVE-26169: Set non-vectorized mode as default when accessing iceberg tables in avro...
László Pintér [Tue, 26 Apr 2022 14:06:55 +0000 (16:06 +0200)] 
HIVE-26169: Set non-vectorized mode as default when accessing iceberg tables in avro fileformat. (#3236) (Laszlo Pinter, reviewed by Marton Bod)

4 weeks agoHIVE-26134: Remove Hive on Spark from the main branch (Peter Vary reviewed by Zoltan...
pvary [Tue, 26 Apr 2022 12:15:07 +0000 (14:15 +0200)] 
HIVE-26134: Remove Hive on Spark from the main branch (Peter Vary reviewed by Zoltan Haindrich) (#3201)

4 weeks agoHIVE-26171: HMSHandler get_all_tables method can not retrieve tables from remote...
Butao Zhang [Tue, 26 Apr 2022 08:42:09 +0000 (16:42 +0800)] 
HIVE-26171: HMSHandler get_all_tables method can not retrieve tables from remote database (Butao Zhang reviewed by Peter Vary) (#3238)

4 weeks agoHIVE-26161: Use Hive's ORC dependency version when producing file footer for Iceberg...
Adam Szita [Thu, 21 Apr 2022 18:03:19 +0000 (20:03 +0200)] 
HIVE-26161: Use Hive's ORC dependency version when producing file footer for Iceberg (#3230) (Adam Szita, reviewed by Marton Bod)

4 weeks agoHIVE-26129: Non blocking DROP CONNECTOR (Denys Kuzmenko, reviewed by Naveen Gangam)
Denys Kuzmenko [Thu, 21 Apr 2022 14:27:56 +0000 (16:27 +0200)] 
HIVE-26129: Non blocking DROP CONNECTOR (Denys Kuzmenko, reviewed by Naveen Gangam)

Closes #3173

5 weeks agoHIVE-26156: Iceberg delete writer should handle deleting from old partition specs...
Marton Bod [Thu, 21 Apr 2022 07:32:51 +0000 (09:32 +0200)] 
HIVE-26156: Iceberg delete writer should handle deleting from old partition specs (#3225) (Marton Bod, reviewed by Adam Szita)

5 weeks agoHIVE-26074: PTF Vectorization: BoundaryScanner for varchar. (#3187). (Ayush Saxena...
Ayush Saxena [Wed, 20 Apr 2022 18:25:17 +0000 (23:55 +0530)] 
HIVE-26074: PTF Vectorization: BoundaryScanner for varchar. (#3187). (Ayush Saxena reviewed by Laszlo Bodor)

5 weeks agoHIVE-26137: Optimized transfer of Iceberg residual expressions from AM to execution...
Adam Szita [Wed, 20 Apr 2022 15:46:55 +0000 (17:46 +0200)] 
HIVE-26137: Optimized transfer of Iceberg residual expressions from AM to execution (#3203) (Adam Szita, reviewed by Marton Bod)

5 weeks agoHIVE-21456: Thrift over Http for Hive Metastore (#3105) (Sourabh Goyal, reviewed...
Sourabh Goyal [Wed, 20 Apr 2022 13:33:37 +0000 (06:33 -0700)] 
HIVE-21456: Thrift over Http for Hive Metastore (#3105) (Sourabh Goyal, reviewed by Sai Gantasala, Peter Vary, Naveen and Yongzhi)

* [WIP]HIVE-21456: Thrift over Http for Hive Metastore

Change-Id: Ie610b7351fe6279353c1f781b0602da0f1860443

* Addresses review comments. Also fixes build failure

Change-Id: Idc8dc3448156e7e2715dc9ea979edf007d4d53d4

* fixes test failures

Change-Id: Ibf6210985248f88cf7011b048703e95fd99dee49

* Refactors creation of Binary and HTTP clients in seprate methods in HiveMetastoreClient

Change-Id: Ib080e24eede76104e10458343f85ac746022f16d

* Addresses review comments

Change-Id: I5ec4fb201bd65bc358c38160348b200fc16d730c

* Fixes validation of maxIdleTimeout in metastore http server

Change-Id: I52990b3904cd8d42da9cff9f282e1f099323e3d7

* Disabled HTTP TRACE in embedded jetty server in HMS

Change-Id: Idcdec4ee0ff7d3ded67816cca4505627a1e5b33b

* Addresses review comments

Change-Id: Ie046c512f2095b0d71743a9485620e369dc75b17

* Addresses nits. Adds some more comments

Change-Id: I39b50cd549af62e5d460fa99167c5aab221edaf8

5 weeks agoHIVE-26146: Handle missing hive.acid.key.index in the fixacidkeyindex utility (Alessa...
Alessandro Solimando [Tue, 19 Apr 2022 15:12:17 +0000 (17:12 +0200)] 
HIVE-26146: Handle missing hive.acid.key.index in the fixacidkeyindex utility (Alessandro Solimando, reviewed by Aman Sinha and Karen Coppage)

Closes #3216.

5 weeks agoHIVE-26147: OrcRawRecordMerger throws NPE when hive.acid.key.index is missing for...
Alessandro Solimando [Tue, 19 Apr 2022 15:07:06 +0000 (17:07 +0200)] 
HIVE-26147: OrcRawRecordMerger throws NPE when hive.acid.key.index is missing for an acid file (Alessandro Solimando, reviewed by Aman Sinha and Karen Coppage)

Closes #3219.

5 weeks agoHIVE-25921: Overwrite table metadata for bootstraped tables. (#2993). (Ayush Saxena...
Ayush Saxena [Sun, 17 Apr 2022 06:02:51 +0000 (11:32 +0530)] 
HIVE-25921: Overwrite table metadata for bootstraped tables. (#2993). (Ayush Saxena, reviewed by Denys Kuzmenko)

5 weeks agoHIVE-25416 : Hive metastore memory leak because datanucleus-api-jdo bug (#2555)
shezm [Thu, 14 Apr 2022 15:20:28 +0000 (23:20 +0800)] 
HIVE-25416 : Hive metastore memory leak because datanucleus-api-jdo bug (#2555)

* HIVE-25416 : Hive metastore memory leak cause datanucleus-api-jdo bug
Co-authored-by: 佘志铭 <shezhiming@GIH-L-4622.local>
5 weeks agoHIVE-26141: Fix vector_ptf_part_simple_all_datatypes source file. (#3211). (Ayush...
Ayush Saxena [Thu, 14 Apr 2022 14:42:25 +0000 (20:12 +0530)] 
HIVE-26141: Fix vector_ptf_part_simple_all_datatypes source file. (#3211). (Ayush Saxena reviewed by Laszlo Bodor)

5 weeks agoHIVE-26133: Insert overwrite on Iceberg tables can result in duplicate entries after...
László Pintér [Thu, 14 Apr 2022 13:22:42 +0000 (15:22 +0200)] 
HIVE-26133: Insert overwrite on Iceberg tables can result in duplicate entries after partition evolution (#3202) (Laszlo Pinter, reviewed by Marton Bod and Peter Vary)

6 weeks agoHIVE-26127: INSERT OVERWRITE throws FileNotFound when destination partition is delete...
Yu-Wen Lai [Sat, 9 Apr 2022 00:56:32 +0000 (17:56 -0700)] 
HIVE-26127: INSERT OVERWRITE throws FileNotFound when destination partition is deleted (Yu-Wen Lai, reviewed by Stamatis Zampetakis)

Closes #3198

6 weeks agoHIVE-26139: Encode only '#' characters in HBaseStorageHandler authorization URL ...
Steve Carlin [Tue, 12 Apr 2022 18:32:25 +0000 (11:32 -0700)] 
HIVE-26139: Encode only '#' characters in HBaseStorageHandler authorization URL (Steve Carlin, reviewed by Alessandro Solimando, Stamatis Zampetakis)

Remove the global encoding of the authorization URL since it has some
undesirable side effects.

Closes #3206

6 weeks agoHIVE-26121: Hive transaction rollback should be thread-safe (Denys Kuzmenko, reviewed...
Denys Kuzmenko [Wed, 13 Apr 2022 09:33:13 +0000 (11:33 +0200)] 
HIVE-26121: Hive transaction rollback should be thread-safe (Denys Kuzmenko, reviewed by Peter Vary)

Closes #3181

6 weeks agoHIVE-26092: Fix javadoc errors for the 4.0.0 release (Peter Vary reviewed by Zoltan...
Peter Vary [Wed, 13 Apr 2022 06:28:09 +0000 (08:28 +0200)] 
HIVE-26092: Fix javadoc errors for the 4.0.0 release (Peter Vary reviewed by Zoltan Haindrich) (#3185) (addendum)

6 weeks agodisable flaky mapjoin_memcheck
Zoltan Haindrich [Tue, 12 Apr 2022 17:28:25 +0000 (17:28 +0000)] 
disable flaky mapjoin_memcheck

6 weeks agoHIVE-26092: Fix javadoc errors for the 4.0.0 release (Peter Vary reviewed by Zoltan...
pvary [Tue, 12 Apr 2022 11:52:52 +0000 (13:52 +0200)] 
HIVE-26092: Fix javadoc errors for the 4.0.0 release (Peter Vary reviewed by Zoltan Haindrich) (#3185)

6 weeks agoHIVE-26102: Implement DELETE statements for Iceberg tables (#3131) (Marton Bod, revie...
Marton Bod [Tue, 12 Apr 2022 08:54:05 +0000 (10:54 +0200)] 
HIVE-26102: Implement DELETE statements for Iceberg tables (#3131) (Marton Bod, reviewed by Peter Vary)

6 weeks agoHIVE-25941: Long compilation time of complex query due to analysis for materialized...
Krisztian Kasa [Tue, 12 Apr 2022 08:20:32 +0000 (10:20 +0200)] 
HIVE-25941: Long compilation time of complex query due to analysis for materialized view rewrite (Krisztian Kasa, reviewed by Zoltan Haindrich,  Aman Sinha)

6 weeks agoHIVE-26093 Deduplicate org.apache.hadoop.hive.metastore.annotation package-info.java...
pvary [Mon, 11 Apr 2022 12:04:41 +0000 (14:04 +0200)] 
HIVE-26093 Deduplicate org.apache.hadoop.hive.metastore.annotation package-info.java (Peter Vary reviewed by Stamatis Zampetakis) (#3168)

6 weeks agoHIVE-25492: Major query-based compaction is skipped if partition is empty (Antal...
Antal Sinkovits [Mon, 11 Apr 2022 10:12:54 +0000 (12:12 +0200)] 
HIVE-25492: Major query-based compaction is skipped if partition is empty  (Antal Sinkovits, reviewed by Denys Kuzmenko)

Closes #3157

6 weeks agoHIVE-26096: Select on single column MultiDelimitSerDe table throws AIOBE (#3158)
Naresh P R [Sat, 9 Apr 2022 00:22:47 +0000 (17:22 -0700)] 
HIVE-26096: Select on single column MultiDelimitSerDe table throws AIOBE (#3158)

6 weeks agoHIVE-26020: Set dependency scope for json-path, commons-compiler and janino to runtim...
Stamatis Zampetakis [Wed, 9 Mar 2022 14:24:14 +0000 (15:24 +0100)] 
HIVE-26020: Set dependency scope for json-path, commons-compiler and janino to runtime (Stamatis Zampetakis, reviewed by Alessandro Solimando, Krisztian Kasa)

Closes #3075

6 weeks agoHIVE-26019: Upgrade com.jayway.jsonpath from 2.4.0 to 2.7.0 (Stamatis Zampetakis...
Stamatis Zampetakis [Wed, 9 Mar 2022 13:20:17 +0000 (14:20 +0100)] 
HIVE-26019: Upgrade com.jayway.jsonpath from 2.4.0 to 2.7.0 (Stamatis Zampetakis, reviewed by Alessandro Solimando, Krisztian Kasa)

Closes #3075

6 weeks agoHIVE-26119: Remove unnecessary Exceptions from DDLPlanUtils (Soumyakanti Das, reviewe...
Soumyakanti Das [Wed, 6 Apr 2022 17:49:20 +0000 (10:49 -0700)] 
HIVE-26119: Remove unnecessary Exceptions from DDLPlanUtils (Soumyakanti Das, reviewed by Stamatis Zampetakis)

Closes #3184

7 weeks agoHIVE-26118: [Standalone beeline] Jar name mismatch in assembly (Naveen Gangam, review...
Naveen Gangam [Thu, 7 Apr 2022 08:18:49 +0000 (04:18 -0400)] 
HIVE-26118: [Standalone beeline] Jar name mismatch in assembly (Naveen Gangam, reviewed by Zhihua Deng) (#3180)

7 weeks agoHIVE-25967: Prevent residual expressions from getting serialized in Iceberg splits...
Adam Szita [Wed, 6 Apr 2022 08:36:31 +0000 (10:36 +0200)] 
HIVE-25967: Prevent residual expressions from getting serialized in Iceberg splits (addendum) (#3178) (Adam Szita, reviewed by Peter Vary)

7 weeks agoHIVE-26116: Fix handling of compaction requests originating from aborted dynamic...
veghlaci05 [Wed, 6 Apr 2022 08:01:11 +0000 (10:01 +0200)] 
HIVE-26116: Fix handling of compaction requests originating from aborted dynamic partition queries in Initiator (Laszlo Vegh, reviewed by Peter Vary and Karen Coppage)

Closes #3177.

7 weeks agoHIVE-25728: ParseException while gathering Column Stats (Soumyakanti Das, reviewed...
Soumyakanti Das [Wed, 6 Apr 2022 04:07:46 +0000 (21:07 -0700)] 
HIVE-25728: ParseException while gathering Column Stats (Soumyakanti Das, reviewed by Krisztian Kasa)

7 weeks agoHIVE-25934: Non blocking RENAME PARTITION implementation (Denys Kuzmenko, reviewed...
Denys Kuzmenko [Tue, 5 Apr 2022 13:26:26 +0000 (15:26 +0200)] 
HIVE-25934: Non blocking RENAME PARTITION implementation (Denys Kuzmenko, reviewed by Peter Vary and Rajesh Balamohan)

Closes #3015

7 weeks agoHIVE-26110: Bulk insert into partitioned table creates lots of files in iceberg ...
Adam Szita [Tue, 5 Apr 2022 09:01:56 +0000 (11:01 +0200)] 
HIVE-26110: Bulk insert into partitioned table creates lots of files in iceberg (#3174) (Adam Szita, reviewed by Marton Bod, Peter Vary and Rajesh Balamohan)

7 weeks agoHIVE-26105 : Show columns shows extra values if column comments contains specific...
mahesh kumar behera [Tue, 5 Apr 2022 08:17:37 +0000 (13:47 +0530)] 
HIVE-26105 : Show columns shows extra values if column comments contains specific Chinese character (Mahesh Kumar Behera, Ramesh Kumar Thangarajan)

7 weeks agoHIVE-26109: Upgrade groovy to 2.4.21 due to CVE (#3171)
Yu-Wen [Mon, 4 Apr 2022 16:50:40 +0000 (09:50 -0700)] 
HIVE-26109: Upgrade groovy to 2.4.21 due to CVE (#3171)

(Yu-Wen Lai via Naveen Gangam)

7 weeks agoHIVE-26103: Port Iceberg fixes to the iceberg module (#3164)
pvary [Mon, 4 Apr 2022 11:04:04 +0000 (13:04 +0200)] 
HIVE-26103: Port Iceberg fixes to the iceberg module (#3164)

* Source Iceberg PR - Core: Remove deprecated APIs up to 0.13.0

* Revert "HIVE-25563: Iceberg table operations hang a long time if metadata is missing/corrupted (Adam Szita, reviewed by Marton Bod)" - applying instead  Hive: Limit number of retries when metadata file is missing (#3379)

This reverts commit 7b600fe38f03b9790b193171a65e57f6a6970820.

* Source Iceberg PR - Hive: Limit number of retries when metadata file is missing (#3379)

* Source Iceberg PR - Hive: Fix RetryingMetaStoreClient for Hive 2.1 (#3403)

* Source Iceberg PR - Switch from new HashMap to Maps.newHashMap (#3648)

* Source Iceberg PR - Hive: HiveCatalog should remove HMS stats for certain engines based on config (#3652) - Use the Iceberg config property

* Source Iceberg PR - Core: If status check fails, commit should be unknown (#3717)

* Source Iceberg PR - Build: Add checkstyle rule for instantiating HashMap, HashSet, ArrayList (#3689)

* Source Iceberg PR - Test: Make sure to delete temp folders (#3790)

* Source Iceberg PR - API: Register existing tables in Iceberg HiveCatalog (#3851)

* Source Iceberg PR - Hive: Make Iceberg table filter optional in HiveCatalog (#3908)

* Source Iceberg PR - Core: Add reserved UUID Table Property and Expose in HMS. (#3914)

* Source Iceberg PR - Hive: Known exception should not become CommitStateUnknownException (#4261)

* Source Iceberg PR - Build: Add missing @OverRide annotations (#3654)

7 weeks agoHIVE-26062: Make sure that running 4.0.0-alpha-2 above 4.0.0-alpha-1 HMS DB reports...
dengzh [Sat, 2 Apr 2022 07:46:21 +0000 (15:46 +0800)] 
HIVE-26062: Make sure that running 4.0.0-alpha-2 above 4.0.0-alpha-1 HMS DB reports the correct error (Zhihua Deng reviewed by Peter Vary) (#3138)

7 weeks agoHIVE-26101: Port Iceberg Hive fix - Hive: Avoid recursive listing in HiveCatalog...
pvary [Fri, 1 Apr 2022 11:51:27 +0000 (13:51 +0200)] 
HIVE-26101: Port Iceberg Hive fix - Hive: Avoid recursive listing in HiveCatalog#renameTable (Peter Vary reviewed by Marton Bod) (#3163)

7 weeks agoHIVE-26100: Preparing for 4.0.0-alpha-2 development (Peter Vary reviewed by Stamatis...
pvary [Fri, 1 Apr 2022 08:36:55 +0000 (10:36 +0200)] 
HIVE-26100: Preparing for 4.0.0-alpha-2 development (Peter Vary reviewed by Stamatis Zampetakis) (#3162)

7 weeks agoHIVE-26098 : Duplicate path/Jar in hive.aux.jars.path or hive.reloadable.aux.jars...
mahesh kumar behera [Fri, 1 Apr 2022 07:18:34 +0000 (12:48 +0530)] 
HIVE-26098 : Duplicate path/Jar in hive.aux.jars.path or hive.reloadable.aux.jars.path causing IllegalArgumentException (Mahesh Kumar Behera, reviewed by Ayush Saxena)

7 weeks agoHIVE-26099: Move patched-iceberg packages to org.apache.hive group (Peter Vary review...
pvary [Thu, 31 Mar 2022 18:51:48 +0000 (20:51 +0200)] 
HIVE-26099: Move patched-iceberg packages to org.apache.hive group (Peter Vary reviewed by Marton Bod) (#3161)

8 weeks agoHIVE-26026: Use the new "REFUSED" compaction state where it makes sense (Laszlo Vegh...
veghlaci05 [Thu, 31 Mar 2022 11:57:38 +0000 (13:57 +0200)] 
HIVE-26026: Use the new "REFUSED" compaction state where it makes sense (Laszlo Vegh, reviewed by Denys Kuzmenko and Karen Coppage)

Closes #3126