hbase.git
18 hours agoHBASE-26666 Add native TLS encryption support to RPC server/client (#4666) master
Andor Molnár [Sat, 6 Aug 2022 23:09:54 +0000 (01:09 +0200)] 
HBASE-26666 Add native TLS encryption support to RPC server/client (#4666)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
29 hours agoHBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big...
Duo Zhang [Sat, 6 Aug 2022 12:08:07 +0000 (20:08 +0800)] 
HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big (#4679)

Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
3 days agoHBASE-27271 Addendum use NettyFutureUtils
Duo Zhang [Thu, 4 Aug 2022 15:14:04 +0000 (23:14 +0800)] 
HBASE-27271 Addendum use NettyFutureUtils

3 days agoHBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorr...
LiangJun He [Thu, 4 Aug 2022 14:42:36 +0000 (22:42 +0800)] 
HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect (#4678)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
3 days agoHBASE-27271 BufferCallBeforeInitHandler should ignore the flush request (#4676)
Duo Zhang [Thu, 4 Aug 2022 14:31:58 +0000 (22:31 +0800)] 
HBASE-27271 BufferCallBeforeInitHandler should ignore the flush request (#4676)

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
5 days agoHBASE-27153 Improvements to read-path tracing
Nick Dimiduk [Thu, 23 Jun 2022 12:20:39 +0000 (14:20 +0200)] 
HBASE-27153 Improvements to read-path tracing

Signed-off-by: Andrew Purtell <apurtell@apache.org>
8 days agoHBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI (#4667)
Bryan Beaudreault [Fri, 29 Jul 2022 19:28:28 +0000 (15:28 -0400)] 
HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI (#4667)

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
10 days agoHBASE-27185 Addendum fix TestShadeSaslAuthenticationProvider
Duo Zhang [Thu, 28 Jul 2022 09:20:13 +0000 (17:20 +0800)] 
HBASE-27185 Addendum fix TestShadeSaslAuthenticationProvider

11 days agoHBASE-27247 TestPerTableCFReplication.testParseTableCFsFromConfig is broken because...
chenglei [Wed, 27 Jul 2022 12:48:54 +0000 (20:48 +0800)] 
HBASE-27247 TestPerTableCFReplication.testParseTableCFsFromConfig is broken because of ReplicationPeerConfigUtil.parseTableCFsFromConfig (#4658)

Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
11 days agoHBASE-27185 Addendum fix a nit problem which is found when cherry-picking to branch-2
Duo Zhang [Wed, 27 Jul 2022 01:21:02 +0000 (09:21 +0800)] 
HBASE-27185 Addendum fix a nit problem which is found when cherry-picking to branch-2

11 days agoHBASE-27236 Clean up error-prone warnings in hbase-hbtop (#4649)
Andrew Purtell [Wed, 27 Jul 2022 01:04:14 +0000 (18:04 -0700)] 
HBASE-27236 Clean up error-prone warnings in hbase-hbtop (#4649)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
11 days agoHBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624)
Duo Zhang [Wed, 27 Jul 2022 01:00:42 +0000 (09:00 +0800)] 
HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
11 days agoHBASE-27210 Clean up error-prone findings in hbase-endpoint (#4646)
Andrew Purtell [Wed, 27 Jul 2022 00:33:35 +0000 (17:33 -0700)] 
HBASE-27210 Clean up error-prone findings in hbase-endpoint (#4646)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
11 days agoHBASE-27203 Clean up error-prone findings in hbase-client (#4644)
Andrew Purtell [Tue, 26 Jul 2022 22:08:24 +0000 (15:08 -0700)] 
HBASE-27203 Clean up error-prone findings in hbase-client (#4644)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
11 days agoHBASE-27087 TestQuotaThrottle times out
Andrew Purtell [Tue, 26 Jul 2022 21:05:19 +0000 (14:05 -0700)] 
HBASE-27087 TestQuotaThrottle times out

Signed-off-by: Andrew Purtell <apurtell@apache.org>
11 days agoHBASE-27229 BucketCache statistics should not count evictions by hfile (#4639)
Bryan Beaudreault [Tue, 26 Jul 2022 20:59:57 +0000 (16:59 -0400)] 
HBASE-27229 BucketCache statistics should not count evictions by hfile (#4639)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
12 days agoHBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634)
Duo Zhang [Tue, 26 Jul 2022 15:42:37 +0000 (23:42 +0800)] 
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
12 days agoHBASE-27225 Add BucketAllocator bucket size statistic logging (#4637)
Bryan Beaudreault [Tue, 26 Jul 2022 00:54:40 +0000 (20:54 -0400)] 
HBASE-27225 Add BucketAllocator bucket size statistic logging (#4637)

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
12 days agoHBASE-27235 Clean up error-prone warnings in hbase-hadoop-compat (#4648)
Andrew Purtell [Tue, 26 Jul 2022 00:30:15 +0000 (17:30 -0700)] 
HBASE-27235 Clean up error-prone warnings in hbase-hadoop-compat (#4648)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
13 days agoHBASE-27179 Issues building with OpenJDK 17 (#4594)
Ramón García Fernández [Mon, 25 Jul 2022 11:21:21 +0000 (13:21 +0200)] 
HBASE-27179 Issues building with OpenJDK 17 (#4594)

Fix test case failures in org.apache.hadoop.hbase.http.log.TestLogLevel under Openjdk 17 because of a missing export of java.security.jgss/sun.security.krb5.
Removed option --illegal-access=permit ignored since Openjdk 17.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
13 days agoHBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640)
Wellington Ramos Chevreuil [Mon, 25 Jul 2022 10:17:16 +0000 (11:17 +0100)] 
HBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640)

Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
13 days agoHBASE-27204 BlockingRpcClient will hang for 20 seconds when SASL is enabled after...
Andrew Purtell [Mon, 25 Jul 2022 06:06:31 +0000 (23:06 -0700)] 
HBASE-27204 BlockingRpcClient will hang for 20 seconds when SASL is enabled after finishing negotiation (#4642)

Revert "HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)"

This reverts commit bd79c4065ccb13a5e217d844376b3e7b9489d2fe.

When Kerberos authentication succeeds, on the server side, after
receiving the final SASL token from the client, we simply wait for
the client to continue by sending the connection header.
After HBASE-24579, on the client side, an additional readStatus()
was added, which mistakenly assumes that after negotiation has
completed a status code will be sent. However when authentication
has succeeded the server will not send one. As a result the client
will hang and only throw an exception when the configured read
timeout is reached, which is 20 seconds by default.

We cannot unilaterally send the expected additional status code
from the server side because older clients will not expect it. The
first call will fail because the client finds unexpected bytes in
the stream ahead of the call response. Fabricating a call response
also does not seem a viable strategy for backwards compatibility.

The HBASE-24579 change needs to be reconsidered given the
difficult backwards compatibility challenges here.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2 weeks agoHBASE-27230 RegionServer should be aborted when WAL.sync throws Timeo… (#4641)
chenglei [Sun, 24 Jul 2022 07:08:03 +0000 (15:08 +0800)] 
HBASE-27230 RegionServer should be aborted when WAL.sync throws Timeo… (#4641)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2 weeks agoHBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup...
Duo Zhang [Thu, 21 Jul 2022 06:26:52 +0000 (14:26 +0800)] 
HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
2 weeks agoRevert "HBASE-27152 Under compaction mark may leak (#4568)"
Duo Zhang [Thu, 21 Jul 2022 01:02:40 +0000 (09:02 +0800)] 
Revert "HBASE-27152 Under compaction mark may leak (#4568)"

This reverts commit da27a67a1e3abfa0117b0cf1a59d01b7a2f8ab05.

2 weeks agoHBASE-27152 Under compaction mark may leak (#4568)
Xiaolin Ha [Wed, 20 Jul 2022 00:49:10 +0000 (08:49 +0800)] 
HBASE-27152 Under compaction mark may leak (#4568)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2 weeks agoHBASE-27219 Change JONI encoding in RegexStringComparator (#4632)
Minwoo Kang [Tue, 19 Jul 2022 16:10:38 +0000 (01:10 +0900)] 
HBASE-27219 Change JONI encoding in RegexStringComparator (#4632)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2 weeks agoHBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
Duo Zhang [Tue, 19 Jul 2022 01:59:22 +0000 (09:59 +0800)] 
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2 weeks agoHBASE-27208 Use spotless to purge the missing summary warnings from error prone ...
Duo Zhang [Tue, 19 Jul 2022 01:45:06 +0000 (09:45 +0800)] 
HBASE-27208 Use spotless to purge the missing summary warnings from error prone (#4628)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2 weeks agoHBASE-27195 Clean up netty worker/thread pool configuration (#4619)
Andrew Purtell [Mon, 18 Jul 2022 22:48:40 +0000 (15:48 -0700)] 
HBASE-27195 Clean up netty worker/thread pool configuration (#4619)

The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2 weeks agoHBASE-27202 Clean up error-prone findings in hbase-balancer (#4623)
Andrew Purtell [Mon, 18 Jul 2022 22:35:43 +0000 (15:35 -0700)] 
HBASE-27202 Clean up error-prone findings in hbase-balancer (#4623)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2 weeks agoHBASE-27200 Clean up error-prone findings in hbase-archetypes (#4621)
Andrew Purtell [Mon, 18 Jul 2022 22:31:34 +0000 (15:31 -0700)] 
HBASE-27200 Clean up error-prone findings in hbase-archetypes (#4621)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2 weeks agoHBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure (#4630)
Duo Zhang [Mon, 18 Jul 2022 11:25:06 +0000 (19:25 +0800)] 
HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure (#4630)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
3 weeks agoHBASE-27144 Add special rpc handlers for bulkload operations (#4558)
SiCheng-Zheng [Sun, 17 Jul 2022 13:21:58 +0000 (21:21 +0800)] 
HBASE-27144 Add special rpc handlers for bulkload operations (#4558)

Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
3 weeks agoHBASE-27053 IOException during caching of uncompressed block to the block cache ...
Bryan Beaudreault [Sat, 16 Jul 2022 21:30:55 +0000 (17:30 -0400)] 
HBASE-27053 IOException during caching of uncompressed block to the block cache (#4610)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
3 weeks agoHBASE-27207 ConnectionUtils.allOf should be moved to FutureUtils (#4627)
chenglei [Fri, 15 Jul 2022 09:04:03 +0000 (17:04 +0800)] 
HBASE-27207 ConnectionUtils.allOf should be moved to FutureUtils (#4627)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
3 weeks agoAmend HBASE-27194 Add test coverage for SimpleRpcServer (#4616)
Andrew Purtell [Wed, 13 Jul 2022 18:23:07 +0000 (11:23 -0700)] 
Amend HBASE-27194 Add test coverage for SimpleRpcServer (#4616)

When backporting to branch-2 it was discovered that the order of tear
down operations requires the KDC to be torn down first, although on
master branch the original code did not produce the same errors (a NPE
in KDC#stop). Forward port the difference for maintainability.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
3 weeks agoHBASE-27194 Add test coverage for SimpleRpcServer (#4616)
Andrew Purtell [Wed, 13 Jul 2022 16:28:30 +0000 (09:28 -0700)] 
HBASE-27194 Add test coverage for SimpleRpcServer (#4616)

Add test coverage for SimpleRpcServer.

Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.

Add secure test coverage for both SimpleRpcServer and NettyRpcServer.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
3 weeks agoHBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488)
Andrew Purtell [Wed, 13 Jul 2022 16:01:21 +0000 (09:01 -0700)] 
HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements (#4488)

* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements

- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
  replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
  content and updating a cf that tracks referrer counts. These are not
  validated at this time. It could be possible to log the increments, sum
  them with a reducer, and then verify the total, but this is left as a
  future exercise.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.

* Fix an issue with completion chaining

* Pause loading if too many operations are in flight

3 weeks agoHBASE-27193 TestZooKeeper is flaky (#4615)
Duo Zhang [Wed, 13 Jul 2022 02:32:01 +0000 (10:32 +0800)] 
HBASE-27193 TestZooKeeper is flaky (#4615)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
3 weeks agoHBASE-27161 Improve TestMultiRespectsLimits (#4586)
Andrew Purtell [Tue, 12 Jul 2022 18:19:35 +0000 (11:19 -0700)] 
HBASE-27161 Improve TestMultiRespectsLimits (#4586)

TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
3 weeks agoHBASE-27097 SimpleRpcServer is broken (#4613)
Andrew Purtell [Tue, 12 Jul 2022 18:07:23 +0000 (11:07 -0700)] 
HBASE-27097 SimpleRpcServer is broken (#4613)

Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.

SimpleRpcServer and its related code is now also marked as @Deprecated.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
3 weeks agoHBASE-27192 The retry number for TestSeparateClientZKCluster is too small (#4614)
Duo Zhang [Tue, 12 Jul 2022 11:08:41 +0000 (19:08 +0800)] 
HBASE-27192 The retry number for TestSeparateClientZKCluster is too small (#4614)

Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
3 weeks agoHBASE-27149 Server should close scanner if client times out before results are ready...
Bryan Beaudreault [Tue, 12 Jul 2022 02:22:00 +0000 (22:22 -0400)] 
HBASE-27149 Server should close scanner if client times out before results are ready (#4604)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
3 weeks agoHBASE-27188 Report maxStoreFileCount in jmx (#4609)
Bryan Beaudreault [Tue, 12 Jul 2022 02:11:58 +0000 (22:11 -0400)] 
HBASE-27188 Report maxStoreFileCount in jmx (#4609)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
3 weeks agoHBASE-27186 Report block cache size metrics separately for L1 and L2 (#4608)
Bryan Beaudreault [Tue, 12 Jul 2022 02:00:04 +0000 (22:00 -0400)] 
HBASE-27186 Report block cache size metrics separately for L1 and L2 (#4608)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
3 weeks agoHBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel...
Duo Zhang [Tue, 12 Jul 2022 01:40:19 +0000 (09:40 +0800)] 
HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed (#4611)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
3 weeks agoHBASE-27190 Add some docs about exporting and importing snapshots using Aliyun object...
Jinhu Wu [Mon, 11 Jul 2022 13:07:11 +0000 (21:07 +0800)] 
HBASE-27190 Add some docs about exporting and importing snapshots using Aliyun object storage service(OSS) (#4612)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
3 weeks agoHBASE-27182 Rework tracing configuration
Nick Dimiduk [Mon, 11 Jul 2022 12:37:05 +0000 (14:37 +0200)] 
HBASE-27182 Rework tracing configuration

* Take advantage of the fact that OpenTelemetry can read its configuration from environment
  variables and make use of this where possible, only falling back to passing properties into the
  process launch configuration when it's necessary. DRY up tracing configuration and make it
  easier to manage in a container environment.
* Replace `HBASE_TRACE_OPTS`, which used to act as both a feature flag and a baseline for
  configuration shared across processes. Instead, use `HBASE_OTEL_TRACING_ENABLED` as a feature
  flag, and let configuration reuse be handled via the environment variables that otel supports
  naively.
* Add further explanation for how to write your configuration for our different deployment
  modes (standalone, pseudo-distributed, fully distributed) and in different environments.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
4 weeks agoHBASE-27180 Fix multiple possible buffer leaks (#4597)
Norman Maurer [Fri, 8 Jul 2022 00:19:45 +0000 (02:19 +0200)] 
HBASE-27180 Fix multiple possible buffer leaks (#4597)

* Fix multiple possible buffer leaks

Motivation:

When using ByteBuf you need to be very careful about releasing it as otherwise you might leak data. There were various places in the code-base where such a leak could happen.

Modifications:

- Fix possible buffer leaks
- Ensure we call touch(...) so its easier to debug buffer leaks

Result:

Fix buffer leaks

* Formatting

* Revert some changes as requested

* revert touch

* Also release checksum and header buffers

Signed-off-by: Duo Zhang <zhangduo@apache.org>
4 weeks agoHBASE-18045 Added default SSH connection timeout in Integration tests (#4450)
Narasimha Sharma [Thu, 7 Jul 2022 17:17:04 +0000 (22:47 +0530)] 
HBASE-18045 Added default SSH connection timeout in Integration tests (#4450)

4 weeks agoHBASE-27078 Allow configuring a separate timeout for meta scans (#4557)
Bryan Beaudreault [Thu, 7 Jul 2022 14:02:14 +0000 (10:02 -0400)] 
HBASE-27078 Allow configuring a separate timeout for meta scans (#4557)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
4 weeks agoHBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffe...
Duo Zhang [Thu, 7 Jul 2022 07:55:26 +0000 (15:55 +0800)] 
HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation (#4596)

Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
4 weeks agoHBASE-27157 Addendum fix compile error after revert
Duo Zhang [Thu, 7 Jul 2022 02:26:13 +0000 (10:26 +0800)] 
HBASE-27157 Addendum fix compile error after revert

4 weeks agoRevert "HBASE-27157 Potential race condition in WorkerAssigner (#4577)"
Duo Zhang [Thu, 7 Jul 2022 00:36:21 +0000 (08:36 +0800)] 
Revert "HBASE-27157 Potential race condition in WorkerAssigner (#4577)"

This reverts commit f76d8554ca586f925a1514060e6437fcb0e477d6.

4 weeks agoHBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level (#4593)
Ujjawal [Wed, 6 Jul 2022 23:32:52 +0000 (05:02 +0530)] 
HBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level (#4593)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
4 weeks agoHBASE-27048 Server side scanner time limit should account for time in queue (#4562)
Bryan Beaudreault [Wed, 6 Jul 2022 15:57:37 +0000 (11:57 -0400)] 
HBASE-27048 Server side scanner time limit should account for time in queue (#4562)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
4 weeks agoHBASE-27172 Upgrade OpenTelemetry dependency to 1.15.0
Nick Dimiduk [Wed, 6 Jul 2022 11:54:41 +0000 (13:54 +0200)] 
HBASE-27172 Upgrade OpenTelemetry dependency to 1.15.0

- the agent jar dropped the `-all` classifier after 1.8.0

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
4 weeks agoHBASE-27101 support commons-crypto version 1.1.0 (#4506)
wenwj0 [Wed, 6 Jul 2022 10:57:15 +0000 (18:57 +0800)] 
HBASE-27101 support commons-crypto version 1.1.0 (#4506)

Co-authored-by: wenwj0 <wenweijian2@huawei.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
4 weeks agoHBASE-27148 Addendum uncomment the shutdown code in TestVerifyReplicationCrossDiffHdfs
Duo Zhang [Wed, 6 Jul 2022 06:48:08 +0000 (14:48 +0800)] 
HBASE-27148 Addendum uncomment the shutdown code in TestVerifyReplicationCrossDiffHdfs

4 weeks agoHBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561)
Duo Zhang [Wed, 6 Jul 2022 04:49:47 +0000 (12:49 +0800)] 
HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
4 weeks agoHBASE-27157 Potential race condition in WorkerAssigner (#4577)
Ruanhui [Wed, 6 Jul 2022 02:59:13 +0000 (10:59 +0800)] 
HBASE-27157 Potential race condition in WorkerAssigner (#4577)

Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
4 weeks agoHBASE-27171 Fix Annotation Error in HRegionFileSystem (#4588)
tianhang [Wed, 6 Jul 2022 02:58:34 +0000 (10:58 +0800)] 
HBASE-27171 Fix Annotation Error in HRegionFileSystem (#4588)

Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
4 weeks agoHBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoir...
Bryan Beaudreault [Mon, 4 Jul 2022 20:18:45 +0000 (16:18 -0400)] 
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse (#4592)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
5 weeks agoHBASE-27174 Update downloads.xml for release 2.4.13
Andrew Purtell [Fri, 1 Jul 2022 19:54:21 +0000 (12:54 -0700)] 
HBASE-27174 Update downloads.xml for release 2.4.13

Signed-off-by: Andrew Purtell <apurtell@apache.org>
5 weeks agoHBASE-27169 TestSeparateClientZKCluster is flaky (#4587)
Duo Zhang [Fri, 1 Jul 2022 14:09:01 +0000 (22:09 +0800)] 
HBASE-27169 TestSeparateClientZKCluster is flaky (#4587)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
5 weeks agoHBASE-26218 Add logs in Canary tool (#4501)
Ishika [Thu, 30 Jun 2022 13:53:20 +0000 (19:23 +0530)] 
HBASE-26218 Add logs in Canary tool (#4501)

Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
5 weeks agoHBASE-27164 Add 3.0.0-alpha-3 to download page (#4583)
Duo Zhang [Wed, 29 Jun 2022 03:07:33 +0000 (11:07 +0800)] 
HBASE-27164 Add 3.0.0-alpha-3 to download page (#4583)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
5 weeks agoHBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582)
Duo Zhang [Wed, 29 Jun 2022 03:04:21 +0000 (11:04 +0800)] 
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
5 weeks agoHBASE-27158 Add namespace column family to UNDELETABLE_META_COLUMNFAMILIES (#4578)
Ruanhui [Wed, 29 Jun 2022 02:27:53 +0000 (10:27 +0800)] 
HBASE-27158 Add namespace column family to UNDELETABLE_META_COLUMNFAMILIES (#4578)

Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
5 weeks agoHBASE-27166 WAL value compression minor improvements (#4584)
Andrew Purtell [Tue, 28 Jun 2022 19:49:25 +0000 (12:49 -0700)] 
HBASE-27166 WAL value compression minor improvements (#4584)

A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.

Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
6 weeks agoHBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the...
Duo Zhang [Sun, 26 Jun 2022 07:22:25 +0000 (15:22 +0800)] 
HBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the loop when deletion is succeeded (#4579)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
6 weeks agoRevert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted"
Andrew Purtell [Sat, 25 Jun 2022 17:58:25 +0000 (10:58 -0700)] 
Revert "HBASE-26856 BufferedDataBlockEncoder.OnheapDecodedCell value can get corrupted"

This reverts commit f526609331ba5d6c2377135fdac3ff1dbe71c175.

See also HBASE-27151 TestMultiRespectsLimits.testBlockMultiLimits repeatable failure

6 weeks agoHBASE-27025 Change Hbase book's description for 74.7.3. Load Balancing META table...
huaxiangsun [Fri, 24 Jun 2022 15:01:24 +0000 (08:01 -0700)] 
HBASE-27025 Change Hbase book's description for 74.7.3. Load Balancing META table load (#4530)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
6 weeks agoHBASE-27060 Allow sharing connections between AggregationClient instances (#4566)
Bryan Beaudreault [Fri, 24 Jun 2022 14:47:33 +0000 (10:47 -0400)] 
HBASE-27060 Allow sharing connections between AggregationClient instances (#4566)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
6 weeks agoHBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator (#4571)
Duo Zhang [Fri, 24 Jun 2022 14:29:17 +0000 (22:29 +0800)] 
HBASE-27146 Avoid CellUtil.cloneRow in MetaCellComparator (#4571)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
6 weeks agoHBASE-26095 Modify our ref guide to mention the deprecation of HBTU and also how...
Duo Zhang [Fri, 24 Jun 2022 04:45:00 +0000 (12:45 +0800)] 
HBASE-26095 Modify our ref guide to mention the deprecation of HBTU and also how to make use of the new TestingHBaseCluster (#4542)

Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
6 weeks agoHBASE-27111 Make Netty channel bytebuf allocator configurable. (#4525)
Andrew Purtell [Thu, 23 Jun 2022 22:13:29 +0000 (15:13 -0700)] 
HBASE-27111 Make Netty channel bytebuf allocator configurable. (#4525)

Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
   preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
  name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
  This allows the user to add a custom implementation, perhaps for debugging.

Also updates ReflectionUtils with a new helper method.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
6 weeks agoHBASE-27001 The deleted variable cannot be printed out (#4479)
Kerasone [Thu, 23 Jun 2022 17:11:32 +0000 (01:11 +0800)] 
HBASE-27001 The deleted variable cannot be printed out (#4479)

* The deleted variable cannot be printed out, add it

Co-authored-by: selina.yan <selina.yan@huolala.cn>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
6 weeks agoHBASE-26945 Quotas causes too much load on meta for large clusters (#4567)
Bryan Beaudreault [Thu, 23 Jun 2022 14:34:29 +0000 (10:34 -0400)] 
HBASE-26945 Quotas causes too much load on meta for large clusters (#4567)

Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
6 weeks agoHBASE-26790 Addendum ensure test is not flaky due to async caching
Bryan Beaudreault [Thu, 23 Jun 2022 14:22:55 +0000 (10:22 -0400)] 
HBASE-26790 Addendum ensure test is not flaky due to async caching

6 weeks agoHBASE-26790 getAllRegionLocations can cache locations with null hostname (#4565)
Bryan Beaudreault [Thu, 23 Jun 2022 12:24:49 +0000 (08:24 -0400)] 
HBASE-26790 getAllRegionLocations can cache locations with null hostname (#4565)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
6 weeks agoHBASE-27105 HBaseInterClusterReplicationEndpoint should honor replication adaptive...
Pankaj [Thu, 23 Jun 2022 07:21:48 +0000 (12:51 +0530)] 
HBASE-27105 HBaseInterClusterReplicationEndpoint should honor replication adaptive timeout (#4515)

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
6 weeks agoHBASE-27141 Upgrade hbase-thirdparty dependency to 4.1.1 (#4552)
Duo Zhang [Wed, 22 Jun 2022 14:38:12 +0000 (22:38 +0800)] 
HBASE-27141 Upgrade hbase-thirdparty dependency to 4.1.1 (#4552)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
6 weeks agoHBASE-27062 Remove ThreadPool in HBaseInterClusterReplicationEndpoint when use AsyncC...
chenglei [Wed, 22 Jun 2022 12:01:56 +0000 (20:01 +0800)] 
HBASE-27062 Remove ThreadPool in HBaseInterClusterReplicationEndpoint when use AsyncClusterConnection (#4560)

6 weeks agoRevert "HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (...
chenglei [Wed, 22 Jun 2022 11:38:48 +0000 (19:38 +0800)] 
Revert "HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (#4463)" (#4559)

6 weeks agoHBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (#4463)
chenglei [Wed, 22 Jun 2022 11:33:05 +0000 (19:33 +0800)] 
HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (#4463)

* HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplicationEndpoint when use AsyncClusterConnection

6 weeks agoHBASE-27098 Fix link for field comments (#4499)
litao [Wed, 22 Jun 2022 01:29:34 +0000 (09:29 +0800)] 
HBASE-27098 Fix link for field comments (#4499)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
6 weeks agoHBASE-27143 Add hbase-unsafe as a dependency for a MR job triggered by hbase shell...
Tak Lon (Stephen) Wu [Tue, 21 Jun 2022 20:38:09 +0000 (13:38 -0700)] 
HBASE-27143 Add hbase-unsafe as a dependency for a MR job triggered by hbase shell (#4554)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
6 weeks agoHBASE-26969:Eliminate MOB renames when SFT is enabled (#4418)
BukrosSzabolcs [Tue, 21 Jun 2022 09:18:55 +0000 (11:18 +0200)] 
HBASE-26969:Eliminate MOB renames when SFT is enabled (#4418)

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
6 weeks agoHBASE-27099 The log printing fspread/fsread cost time unit should be milliseconds...
xiaozhang0319 [Mon, 20 Jun 2022 16:13:33 +0000 (00:13 +0800)] 
HBASE-27099 The log printing fspread/fsread cost time unit should be milliseconds (#4500)

Co-authored-by: xiaozhang <issac.zhang@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
6 weeks agoHBASE-27128 when open archiveRetries totalLogSize calculation mistake (#4546)
SiCheng-Zheng [Mon, 20 Jun 2022 15:45:35 +0000 (23:45 +0800)] 
HBASE-27128 when open archiveRetries totalLogSize calculation mistake (#4546)

Co-authored-by: zhengsicheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5833756e5e7cdc01d50538e5c9f959a79fcfd7a3)

7 weeks agoHBASE-27125 The batch size of cleaning expired mob files should have an upper bound...
Xiaolin Ha [Fri, 17 Jun 2022 15:35:26 +0000 (23:35 +0800)] 
HBASE-27125 The batch size of cleaning expired mob files should have an upper bound (#4541)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
7 weeks agoHBASE-27117 Update the method comments for RegionServerAccounting (#4532)
litao [Fri, 17 Jun 2022 00:18:18 +0000 (08:18 +0800)] 
HBASE-27117 Update the method comments for RegionServerAccounting (#4532)

7 weeks agoHBASE-26956 ExportSnapshot tool supports removing TTL (#4351) 3.0.0-alpha-3RC0 rel/3.0.0-alpha-3
XinSun [Wed, 15 Jun 2022 07:04:17 +0000 (15:04 +0800)] 
HBASE-26956 ExportSnapshot tool supports removing TTL (#4351)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
7 weeks agoHBASE-26167 Allow users to not start zookeeper and dfs cluster when using TestingHBas...
xicm [Wed, 15 Jun 2022 02:48:48 +0000 (10:48 +0800)] 
HBASE-26167 Allow users to not start zookeeper and dfs cluster when using TestingHBaseCluster (#4534)

Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>
7 weeks agoRevert "HBASE-26167 Allow users to not start zookeeper and dfs cluster when using...
Duo Zhang [Wed, 15 Jun 2022 02:53:48 +0000 (10:53 +0800)] 
Revert "HBASE-26167 Allow users to not start zookeeper and dfs cluster when using TestingHBaseCluster (#4534)"

This reverts commit 11ae7208dd18ffcf8cd3631f5ca190ad19651209.

7 weeks agoHBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0...
Duo Zhang [Wed, 15 Jun 2022 02:51:31 +0000 (10:51 +0800)] 
HBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0-alpha-3 (#4535)

Signed-off-by: Xin Sun <ddupgs@gmail.com>
7 weeks agoHBASE-26167 Allow users to not start zookeeper and dfs cluster when using TestingHBas...
Duo Zhang [Wed, 15 Jun 2022 02:48:48 +0000 (10:48 +0800)] 
HBASE-26167 Allow users to not start zookeeper and dfs cluster when using TestingHBaseCluster (#4534)

Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>
7 weeks agoHBASE-27091 Speed up the loading of table descriptor from filesystem (#4493)
LiangJun He [Sat, 11 Jun 2022 00:48:23 +0000 (08:48 +0800)] 
HBASE-27091 Speed up the loading of table descriptor from filesystem (#4493)

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>