Viraj Jasani [Sat, 21 May 2022 01:28:59 +0000 (18:28 -0700)]
HDFS-16582. Expose aggregate latency of slow node as perceived by the reporting node (#4323)
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
slfan1989 [Fri, 20 May 2022 20:19:42 +0000 (13:19 -0700)]
YARN-10465. Support getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation (#4317)
Viraj Jasani [Fri, 20 May 2022 18:21:13 +0000 (11:21 -0700)]
HADOOP-18224. Upgrade maven compiler plugin to 3.10.1 (#4267)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Fri, 20 May 2022 17:53:14 +0000 (18:53 +0100)]
HDFS-16453. Upgrade okhttp from 2.7.5 to 4.9.3 (#4229)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
9uapaw [Fri, 20 May 2022 08:08:19 +0000 (10:08 +0200)]
HADOOP-18120. Hadoop auth does not handle HTTP Headers in a case-insensitive way. Contributed by Janos Makai.
Ritesh H Shukla [Thu, 19 May 2022 20:20:24 +0000 (13:20 -0700)]
HADOOP-18245 Extend KMS related exceptions that get mapped to ConnectException (#4329)
9uapaw [Mon, 16 May 2022 08:40:46 +0000 (10:40 +0200)]
YARN-11152. QueueMetrics is leaking memory when creating a new queue during reinitialisation
9uapaw [Wed, 11 May 2022 16:07:04 +0000 (18:07 +0200)]
YARN-11147. ResourceUsage and QueueCapacities classes provide node label iterators that are not thread safe
Hexiaoqiao [Wed, 18 May 2022 15:28:45 +0000 (23:28 +0800)]
HDFS-16540. Addendum: Data locality is lost when DataNode pod restarts in kubernetes. (#4326)
slfan1989 [Wed, 18 May 2022 11:12:04 +0000 (04:12 -0700)]
HADOOP-18229. Fix Hadoop-Common JavaDoc Errors (#4292)
Contributed by slfan1989
Ashutosh Gupta [Tue, 17 May 2022 19:34:12 +0000 (20:34 +0100)]
HADOOP-18237. Upgrade Apache Xerces Java to 2.12.2 (#4318)
Upgrade Apache Xerces Java to 2.12.2 due to handle vulnerability CVE-2022-23437
Contributed by Ashutosh Gupta
Steve Loughran [Tue, 17 May 2022 18:00:09 +0000 (19:00 +0100)]
HADOOP-18198. Release 3.3.3: release notes and jdiff files.
* Add the changelog and release notes
* add all jdiff XML files
* update the project pom with the new stable version
Change-Id: Iaea846c3e451bbd446b45de146845a48953d580d
Ashutosh Gupta [Mon, 16 May 2022 16:47:14 +0000 (17:47 +0100)]
HADOOP-18234. Fix s3a access point xml examples (#4309)
Contributed by Ashutosh Gupta
Zilong Zhu [Mon, 16 May 2022 16:40:09 +0000 (00:40 +0800)]
YARN-11133. YarnClient gets the wrong EffectiveMinCapacity value (#4280)
* Update QueueConfigurationsPBImpl.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
* Update TestPBImplRecords.java
Viraj Jasani [Mon, 16 May 2022 14:34:39 +0000 (07:34 -0700)]
HADOOP-18228. Update hadoop-vote to use HADOOP_RC_VERSION dir (#4272)
Signed-off-by: stack@apache.org
Ashutosh Gupta [Mon, 16 May 2022 14:11:17 +0000 (15:11 +0100)]
YARN-11092. Upgrade jquery ui to 1.13.1 (#4260)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
slfan1989 [Sun, 15 May 2022 16:16:06 +0000 (09:16 -0700)]
YARN-11122. Support getClusterNodes API in FederationClientInterceptor (#4274)
Ashutosh Gupta [Fri, 13 May 2022 16:41:06 +0000 (17:41 +0100)]
YARN-10080. Support show app id on localizer thread pool (#4283)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Fri, 13 May 2022 16:34:19 +0000 (17:34 +0100)]
MAPREDUCE-7377. Remove unused imports in MapReduce project (#4299)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Jian Chen [Fri, 13 May 2022 16:11:42 +0000 (09:11 -0700)]
YARN-11073. Avoid unnecessary preemption for tiny queues under certain corner cases (#4110)
Co-authored-by: Jian Chen <jian.chen@airbnb.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Szilard Nemeth [Sun, 8 May 2022 21:05:08 +0000 (23:05 +0200)]
YARN-11123. ResourceManager webapps test failures due to org.apache.hadoop.metrics2.MetricsException and subsequent java.net.BindException: Address already in use. Contributed by Szilard Nemeth
Hui Fei [Fri, 13 May 2022 06:44:41 +0000 (14:44 +0800)]
Revert "HDFS-14750. RBF: Support dynamic handler allocation in routers (#4199)" (#4306)
Felix Nguyen [Fri, 13 May 2022 06:28:53 +0000 (14:28 +0800)]
HDFS-14750. RBF: Support dynamic handler allocation in routers (#4199)
Szilard Nemeth [Thu, 12 May 2022 11:42:06 +0000 (13:42 +0200)]
YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok
singer-bin [Thu, 12 May 2022 08:53:09 +0000 (16:53 +0800)]
HDFS-16525.System.err should be used when error occurs in multiple methods in DFSAdmin class (#4122)
Lei Yang [Mon, 9 May 2022 18:52:15 +0000 (11:52 -0700)]
HADOOP-18193:Support nested mount points in INodeTree
Fixes #4181
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
Gautham B A [Wed, 11 May 2022 17:34:22 +0000 (23:04 +0530)]
HDFS-16465. Remove redundant strings.h inclusions (#4279)
* Remove redundant strings.h inclusions
* strings.h was included in a bunch of
C/C++ files and were redundant.
* Also, strings.h is not available on
Windows and thus isn't cross-platform
compatible.
* Build for all platforms in CI
* Revert "Build for all platforms in CI"
This reverts commit
2650f047bd6791a5908cfbe50cc8e70d42c512cb.
* Debug failure on Centos 8
* Skipping pipeline run on
Centos 7 to debug the
failure on Centos 8.
* Revert "Debug failure on Centos 8"
This reverts commit
e365e34d6fab9df88f4df622910ddb28a8c8796f.
Szilard Nemeth [Wed, 20 Apr 2022 17:39:47 +0000 (19:39 +0200)]
YARN-11114. RMWebServices returns only apps matching exactly the submitted queue name. Contributed by Szilard Nemeth
Szilard Nemeth [Wed, 11 May 2022 14:55:19 +0000 (16:55 +0200)]
MAPREDUCE-7379. RMContainerRequestor#makeRemoteRequest has confusing log message. Contributed by Ashutosh Gupta
Szilard Nemeth [Wed, 11 May 2022 12:39:42 +0000 (14:39 +0200)]
YARN-10850. TimelineService v2 lists containers for all attempts when filtering for one. Contributed by Benjamin Teke
Szilard Nemeth [Wed, 11 May 2022 12:27:34 +0000 (14:27 +0200)]
YARN-11141. Capacity Scheduler does not support ambiguous queue names when moving application across queues. Contributed by Andras Gyori
slfan1989 [Wed, 11 May 2022 06:44:52 +0000 (23:44 -0700)]
YARN-11130. removed unused import (#4276)
Viraj Jasani [Wed, 11 May 2022 00:10:03 +0000 (17:10 -0700)]
HDFS-16568. dfsadmin -reconfig option to start/query reconfig on all live datanodes (#4264)
Signed-off-by: Tao Li <tomscut@apache.org>
hchaverr [Thu, 5 May 2022 19:39:58 +0000 (12:39 -0700)]
HADOOP-18222. Prevent DelegationTokenSecretManagerMetrics from registering multiple times
Fixes #4266
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
daimin [Tue, 10 May 2022 00:01:16 +0000 (08:01 +0800)]
Fix TestDFSStripedInputStreamWithRandomECPolicy (#4291)
Ayush Saxena [Mon, 9 May 2022 17:26:14 +0000 (22:56 +0530)]
MAPREDUCE-7376. AggregateWordCount fetches wrong results. (#4257). Contributed by Ayush Saxena.
Reviewed-by: Steve Loughran <stevel@apache.org>
Gautham B A [Sun, 8 May 2022 18:17:13 +0000 (23:47 +0530)]
HDFS-16572. Fix typo in readme of hadoop-project-dist
Ashutosh Gupta [Sat, 7 May 2022 23:09:24 +0000 (00:09 +0100)]
HADOOP-17479. Fix the examples of hadoop config prefix (#4197)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Sat, 7 May 2022 23:05:34 +0000 (00:05 +0100)]
HDFS-16185. Fix comment in LowRedundancyBlocks.java (#4194)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Sat, 7 May 2022 22:38:32 +0000 (23:38 +0100)]
HADOOP-16515. Update the link to compatibility guide (#4226)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
daimin [Fri, 6 May 2022 17:30:32 +0000 (01:30 +0800)]
HDFS-16520. Improve EC pread: avoid potential reading whole block (#4104)
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Gautham B A [Wed, 4 May 2022 14:49:08 +0000 (20:19 +0530)]
HDFS-16564. Use uint32_t for hdfs_find (#4245)
* hdfs_find uses u_int32_t type for
storing the value for the max-depth
command line argument.
* The type u_int32_t isn't standard,
isn't available on Windows and thus
breaks cross-platform compatibility.
We need to replace this with uint32_t
which is available on all platforms since
it's part of the C++ standard.
Daniel Carl Jones [Tue, 3 May 2022 13:26:52 +0000 (14:26 +0100)]
HADOOP-18168. Fix S3A ITestMarkerTool use of purged public bucket. (#4140)
This moves off use of the purged s3a://landsat-pds bucket, so fixing tests
which had started failing.
* Adds a new class, PublicDatasetTestUtils to manage the use of public datasets.
* The new test bucket s3a://usgs-landsat/ is requester pays, so depends upon
HADOOP-14661.
Consult the updated test documentation when running against other S3 stores.
Contributed by Daniel Carl Jones
Change-Id: Ie8585e4d9b67667f8cb80b2970225d79a4f8d257
Steve Loughran [Tue, 3 May 2022 13:27:52 +0000 (14:27 +0100)]
Revert "HADOOP-18168. . (#4140)"
This reverts commit
6ab7b72cd61a16f7bcf0d7cae15d1d13b04e1eb1.
Daniel Carl Jones [Tue, 3 May 2022 13:26:52 +0000 (14:26 +0100)]
HADOOP-18168. . (#4140)
This moves off use of the purged s3a://landsat-pds bucket, so fixing tests
which had started failing.
* Adds a new class, PublicDatasetTestUtils to manage the use of public datasets.
* The new test bucket s3a://usgs-landsat/ is requester pays, so depends upon
HADOOP-14661.
Consult the updated test documentation when running against other S3 stores.
Contributed by Daniel Carl Jones
Steve Loughran [Wed, 27 Apr 2022 16:31:17 +0000 (17:31 +0100)]
HADOOP-18198. add -mvnargs option to create-release command line (#4239)
This allows for builds to be run with options like
--mvnargs="-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false"
Contributed by Ayush Saxena.
Change-Id: I396e82d0915d679657d063a948f865041bcdde29
Murali Krishna [Tue, 3 May 2022 12:12:45 +0000 (17:42 +0530)]
HDFS-16562. Upgrade moment.min.js to 2.29.2 (#4240)
Viraj Jasani [Mon, 2 May 2022 21:05:40 +0000 (14:05 -0700)]
HDFS-16521. DFS API to retrieve slow datanodes (#4107)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
jteagles [Mon, 2 May 2022 18:14:26 +0000 (13:14 -0500)]
YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class (#4242)
Co-authored-by: Jonathan Eagles <jeagles@verizonmedia.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Mon, 2 May 2022 18:08:30 +0000 (19:08 +0100)]
YARN-10187. Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. (#4222)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Akira Ajisaka [Mon, 2 May 2022 11:25:09 +0000 (20:25 +0900)]
HADOOP-18219. Fix shaded client test failure (#4254)
This patch MUST be cherrypicked into all branches with
HADOOP-15983 Use jersey-json that is built to use jackson2
Contributed by Akira Ajisaka
Ashutosh Gupta [Mon, 2 May 2022 00:08:12 +0000 (01:08 +0100)]
HDFS-16256. Minor fix in HDFS Fedbalance document (#4192)
Gautham B A [Fri, 29 Apr 2022 16:55:09 +0000 (22:25 +0530)]
HDFS-16468. Define ssize_t for Windows (#4228)
* Some C/C++ files use ssize_t data type.
This isn't available for Windows and we
need to define an alias for this and set it
to an appropriate type to make it cross
platform compatible.
smarthan [Fri, 29 Apr 2022 14:59:23 +0000 (22:59 +0800)]
HDFS-16553. Fix checkstyle for the length of BlockManager construction method over limit. (#4211). Contributed by Chengwei Wang.
Viraj Jasani [Fri, 29 Apr 2022 09:00:17 +0000 (02:00 -0700)]
HDFS-16528. Reconfigure slow peer enable for Namenode (#4186)
stack [Thu, 28 Apr 2022 17:28:01 +0000 (10:28 -0700)]
HDFS-16540. Data locality is lost when DataNode pod restarts in kubernetes (#4170)
This reverts the previous commit
4e47eb66d123014b653d2a2c278a35366902d8a0
undone so I could reapply with the '.' after the HDFS-16540 as is done
in all other commits.
stack [Thu, 28 Apr 2022 17:26:46 +0000 (10:26 -0700)]
Revert "HDFS-16540 Data locality is lost when DataNode pod restarts in kubernetes (#4170)"
Revert to add the '.' after HDFS-16540 so commit message format matches
precedent
This reverts commit
bda0881bf90ae64b32dbaf7b42413e18598e434d.
huaxiangsun [Thu, 28 Apr 2022 17:24:37 +0000 (10:24 -0700)]
HDFS-16540 Data locality is lost when DataNode pod restarts in kubernetes (#4170)
When DN with the same UUID is registered with a different IP, host2DatanodeMap needs to be updated accordingly.
PJ Fanning [Thu, 28 Apr 2022 05:18:19 +0000 (07:18 +0200)]
HADOOP-15983. Use jersey-json that is built to use jackson2 (#3988)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Felix Nguyen [Wed, 27 Apr 2022 06:42:30 +0000 (14:42 +0800)]
HDFS-16539. RBF: Support refreshing/changing router fairness policy controller without rebooting router (#4168)
smarthan [Wed, 27 Apr 2022 03:44:38 +0000 (11:44 +0800)]
HDFS-16554. Remove unused configuration dfs.namenode.block.deletion.increment. (#4213). Contributed by Chengwei Wang.
liever18 [Tue, 26 Apr 2022 18:53:11 +0000 (02:53 +0800)]
HADOOP-18216. io.file.buffer.size must be positive (#4220)
Documents that the buffer size must be > 0
Contributed by Jingxuan Fu.
hchaverri [Tue, 26 Apr 2022 16:20:11 +0000 (09:20 -0700)]
HADOOP-18167. Add metrics to track delegation token secret manager op… (#4092)
* HADOOP-18167. Add metrics to track delegation token secret manager operations
litao [Tue, 26 Apr 2022 03:48:06 +0000 (11:48 +0800)]
HDFS-16488. [SPS]: Expose metrics to JMX for external SPS (#4234)
Ashutosh Gupta [Tue, 26 Apr 2022 01:51:11 +0000 (02:51 +0100)]
YARN-10303. Fixing rest api example, status code error and other typos in ResourceManagerRest.md (#4221)
Ashutosh Gupta [Mon, 25 Apr 2022 05:29:35 +0000 (06:29 +0100)]
MAPREDUCE-7246. In MapredAppMasterRest#Mapreduce_Application_Master_Info_API, updating the datatype of appId to "string". (#4223)
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Ashutosh Gupta [Mon, 25 Apr 2022 05:11:41 +0000 (06:11 +0100)]
HDFS-16255. RBF: Fix dead link to fedbalance document (#4193)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Steve Loughran [Sun, 24 Apr 2022 16:30:41 +0000 (17:30 +0100)]
HADOOP-16202. Enhanced openFile(): hadoop-azure changes. (#2584/4)
Stops the abfs connector warning if openFile().withFileStatus()
is invoked with a FileStatus is not an abfs VersionedFileStatus.
Contributed by Steve Loughran.
Change-Id: I85076b365eb30aaef2ed35139fa8714efd4d048e
Steve Loughran [Sun, 24 Apr 2022 16:23:19 +0000 (17:23 +0100)]
HADOOP-16202. Enhanced openFile(): hadoop-aws changes. (#2584/3)
S3A input stream support for the few fs.option.openfile settings.
As well as supporting the read policy option and values,
if the file length is declared in fs.option.openfile.length
then no HEAD request will be issued when opening a file.
This can cut a few tens of milliseconds off the operation.
The patch adds a new openfile parameter/FS configuration option
fs.s3a.input.async.drain.threshold (default: 16000).
It declares the number of bytes remaining in the http input stream
above which any operation to read and discard the rest of the stream,
"draining", is executed asynchronously.
This asynchronous draining offers some performance benefit on seek-heavy
file IO.
Contributed by Steve Loughran.
Change-Id: I9b0626bbe635e9fd97ac0f463f5e7167e0111e39
Steve Loughran [Sun, 24 Apr 2022 16:10:34 +0000 (17:10 +0100)]
HADOOP-16202. Enhanced openFile(): mapreduce and YARN changes. (#2584/2)
These changes ensure that sequential files are opened with the
right read policy, and split start/end is passed in.
As well as offering opportunities for filesystem clients to
choose fetch/cache/seek policies, the settings ensure that
processing text files on an s3 bucket where the default policy
is "random" will still be processed efficiently.
This commit depends on the associated hadoop-common patch,
which must be committed first.
Contributed by Steve Loughran.
Change-Id: Ic6713fd752441cf42ebe8739d05c2293a5db9f94
Steve Loughran [Sun, 24 Apr 2022 16:03:59 +0000 (17:03 +0100)]
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:
fs.option.openfile.read.policy
A list of the desired read policy, in preferred order.
standard values are
adaptive, default, random, sequential, vector, whole-file
fs.option.openfile.length
How long the file is.
fs.option.openfile.split.start
start of a task's split
fs.option.openfile.split.end
end of a task's split
These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data
The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.
Contributed by Steve Loughran.
Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
Hui Fei [Sun, 24 Apr 2022 09:18:31 +0000 (17:18 +0800)]
Revert "HDFS-16488. [SPS]: Expose metrics to JMX for external SPS (#4035)" (#4232)
litao [Sun, 24 Apr 2022 06:54:41 +0000 (14:54 +0800)]
HDFS-16488. [SPS]: Expose metrics to JMX for external SPS (#4035)
daimin [Sat, 23 Apr 2022 03:45:22 +0000 (11:45 +0800)]
HDFS-16519. Add throttler to EC reconstruction (#4101)
Reviewed-by: litao <tomleescut@gmail.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
litao [Sat, 23 Apr 2022 03:14:06 +0000 (11:14 +0800)]
HDFS-16552. Fix NPE for TestBlockManager (#4210)
GuoPhilipse [Fri, 22 Apr 2022 18:01:20 +0000 (02:01 +0800)]
HDFS-16556. Fix typos in distcp (#4217)
Ashutosh Gupta [Fri, 22 Apr 2022 16:59:41 +0000 (17:59 +0100)]
HADOOP-17564. Fix typo in UnixShellGuide.html (#4195)
contributed by Ashutosh Gupta
Junfan Zhang [Fri, 22 Apr 2022 02:37:28 +0000 (10:37 +0800)]
YARN-11111. Recovery failure when node-label configure-type transit from delegated-centralized to centralized (#4200)
Ashutosh Gupta [Thu, 21 Apr 2022 13:16:12 +0000 (14:16 +0100)]
HADOOP-17551. Upgrade maven-site-plugin to 3.11.0 (#4196)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
smarthan [Thu, 21 Apr 2022 04:32:12 +0000 (12:32 +0800)]
HDFS-16500. Make asynchronous blocks deletion lock and unlock durtion threshold configurable. (#4061). Contributed by Chengwei Wang.
S O'Donnell [Wed, 20 Apr 2022 19:34:43 +0000 (20:34 +0100)]
Revert "HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell."
This reverts commit
dbeeee03639f41a022dd07d5fc04e3aa65a94b5f.
Renukaprasad C [Wed, 20 Apr 2022 13:27:43 +0000 (18:57 +0530)]
HDFS-16526. Addendum Add metrics for slow DataNode (#4191)
qinyuren [Wed, 20 Apr 2022 06:04:27 +0000 (14:04 +0800)]
HDFS-16544. EC decoding failed due to invalid buffer (#4179)
Xing Lin [Wed, 20 Apr 2022 05:47:02 +0000 (22:47 -0700)]
HADOOP-18172: Changed scope for isRootInternalDir/getRootFallbackLink for InodeTree (#4106)
* HADOOP-18172: Change scope of InodeTree and its member methods to make them accessible from outside package.
Co-authored-by: Xing Lin <xinglin@linkedin.com>
Ashutosh Gupta [Tue, 19 Apr 2022 05:35:23 +0000 (06:35 +0100)]
HDFS-16035. Remove DummyGroupMapping as it is not longer used anywhere. (#4183)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
qinyuren [Tue, 19 Apr 2022 04:37:28 +0000 (12:37 +0800)]
HDFS-16538. EC decoding failed due to not enough valid inputs (#4167)
Co-authored-by: liubingxing <liubingxing@bigo.sg>
Steve Loughran [Mon, 18 Apr 2022 18:24:35 +0000 (19:24 +0100)]
HADOOP-18202. create-release fails fatal: unsafe repository (#4188)
Since April 2022/CVE-2022-24765, git refuses to work in directories
whose owner != the current user, unless explicitly told to trust it.
This patches the create-release script to trust the /build/source
dir mounted from the hosting OS, whose userid is inevitably different
from that of the account in the container running git.
Contributed by: Steve Loughran, Ayush Saxena and the new git error messages
Happy-shi [Mon, 18 Apr 2022 17:23:57 +0000 (01:23 +0800)]
HDFS-16541. Fix a typo in NameNodeLayoutVersion (#4176)
jianghuazhu [Mon, 18 Apr 2022 03:21:51 +0000 (11:21 +0800)]
HDFS-16389.Improve NNThroughputBenchmark test mkdirs. (#3819)
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Quanlong Huang [Mon, 18 Apr 2022 02:32:29 +0000 (10:32 +0800)]
HDFS-16535. SlotReleaser should reuse the domain socket based on socket paths (#4158)
Reviewed-by: Lisheng Sun <sunlisheng@apache.org>
Stephen O'Donnell [Sun, 17 Apr 2022 12:05:11 +0000 (13:05 +0100)]
HDFS-16531. Avoid setReplication writing an edit record if old replication equals the new value (#4148). Contributed by Stephen O'Donnell.
He Xiaoqiao [Sun, 17 Apr 2022 11:21:27 +0000 (19:21 +0800)]
HDFS-16534. Split FsDatasetImpl from block pool locks to volume grain locks. (#4141) Contributed by limingxiang.
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Renukaprasad C [Fri, 15 Apr 2022 16:07:05 +0000 (21:37 +0530)]
HDFS-16526. Add metrics for slow DataNode (#4162)
Viraj Jasani [Fri, 15 Apr 2022 14:24:02 +0000 (19:54 +0530)]
HADOOP-18196. Remove replace-guava from replacer plugin (#4152)
章锡平 [Fri, 15 Apr 2022 08:50:49 +0000 (16:50 +0800)]
YARN-11107. Addendum. When NodeLabel is enabled for a YARN cluster, AM blacklist program does not work properly. (#4175)
Daniel Carl Jones [Thu, 14 Apr 2022 15:21:34 +0000 (16:21 +0100)]
HADOOP-18201. Remove endpoint config overrides for ITestS3ARequesterPays (#4169)
Contributed by Daniel Carl Jones.
caozhiqiang [Thu, 14 Apr 2022 09:42:39 +0000 (17:42 +0800)]
HDFS-16456. EC: Decommission a rack with only on dn will fail when the rack number is equal with replication (#4126)
daimin [Thu, 14 Apr 2022 03:07:06 +0000 (11:07 +0800)]
HDFS-16509. Fix decommission UnsupportedOperationException (#4077). Contributed by daimin.
Takanobu Asanuma [Thu, 14 Apr 2022 02:23:38 +0000 (11:23 +0900)]
HDFS-16479. EC: NameNode should not send a reconstruction work when the source datanodes are insufficient (#4138)
章锡平 [Fri, 8 Apr 2022 07:19:17 +0000 (15:19 +0800)]
YARN-11107. When NodeLabel is enabled for a YARN cluster, AM blacklist program does not work properly. Contributed by zhangxiping1