hadoop.git
11 months agoHDFS-13252. Code refactoring: Remove Diff.ListType.
Tsz-Wo Nicholas Sze [Fri, 9 Mar 2018 23:25:41 +0000 (15:25 -0800)] 
HDFS-13252. Code refactoring: Remove Diff.ListType.

11 months agoHDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contrib...
Hanisha Koneru [Fri, 9 Mar 2018 23:27:17 +0000 (15:27 -0800)] 
HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham.

11 months agoHDFS-13190. Document WebHDFS support for snapshot diff
Lokesh Jain [Fri, 9 Mar 2018 23:04:14 +0000 (15:04 -0800)] 
HDFS-13190. Document WebHDFS support for snapshot diff

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
11 months agoHDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. YARN-8006
Hanisha Koneru [Fri, 9 Mar 2018 21:20:32 +0000 (13:20 -0800)] 
HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus.

11 months agoHADOOP-15293. TestLogLevel fails on Java 9
Takanobu Asanuma [Fri, 9 Mar 2018 18:20:35 +0000 (10:20 -0800)] 
HADOOP-15293. TestLogLevel fails on Java 9

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoYARN-8000. Yarn Service component instance name shows up as component name in contain...
Billie Rinaldi [Fri, 9 Mar 2018 16:50:28 +0000 (08:50 -0800)] 
YARN-8000. Yarn Service component instance name shows up as component name in container record. Contributed by Chandni Singh

11 months agoMAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both...
Jason Lowe [Fri, 9 Mar 2018 16:41:16 +0000 (10:41 -0600)] 
MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao

11 months agoHADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output.
Steve Loughran [Fri, 9 Mar 2018 10:44:07 +0000 (10:44 +0000)] 
HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output.
Contributed by Steve Loughran.

11 months agoHDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the...
Yiqun Lin [Fri, 9 Mar 2018 07:42:57 +0000 (15:42 +0800)] 
HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan.

11 months agoYARN-7944. [UI2] Remove master node link from headers of application pages. Contribut...
Sunil G [Thu, 8 Mar 2018 18:22:38 +0000 (23:52 +0530)] 
YARN-7944. [UI2] Remove master node link from headers of application pages. Contributed by Yesha Vora.

11 months agoHADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple...
Xiao Chen [Thu, 8 Mar 2018 18:16:37 +0000 (10:16 -0800)] 
HADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk. Contributed by Bharat Viswanadham.

11 months agoHDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active...
Inigo Goiri [Thu, 8 Mar 2018 17:32:05 +0000 (09:32 -0800)] 
HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun.

11 months agoHADOOP-15273.distcp can't handle remote stores with different checksum algorithms.
Steve Loughran [Thu, 8 Mar 2018 11:24:06 +0000 (11:24 +0000)] 
HADOOP-15273.distcp can't handle remote stores with different checksum algorithms.
Contributed by Steve Loughran.

11 months agoHADOOP-15292. Distcp's use of pread is slowing it down.
Steve Loughran [Thu, 8 Mar 2018 11:15:46 +0000 (11:15 +0000)] 
HADOOP-15292. Distcp's use of pread is slowing it down.
Contributed by Virajith Jalaparti.

11 months agoYARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemote...
Weiwei Yang [Thu, 8 Mar 2018 10:13:36 +0000 (18:13 +0800)] 
YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Tao Yang.

11 months agoHADOOP-15296. Fix a wrong link for RBF in the top page. Contributed by Takanobu Asanuma.
Yiqun Lin [Thu, 8 Mar 2018 08:02:34 +0000 (16:02 +0800)] 
HADOOP-15296. Fix a wrong link for RBF in the top page. Contributed by Takanobu Asanuma.

11 months agoYARN-7891. LogAggregationIndexedFileController should support read from HAR file...
Wangda Tan [Wed, 7 Mar 2018 23:46:22 +0000 (15:46 -0800)] 
YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)

Change-Id: I16e081f21c5f80160564c49cc49d103bd8eb7e16

11 months agoRevert "YARN-7891. LogAggregationIndexedFileController should support read from HAR...
Wangda Tan [Wed, 7 Mar 2018 23:42:29 +0000 (15:42 -0800)] 
Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)"

This reverts commit 4d53ef7eefb14661d824924e503a910de1ae997f.

11 months agoHDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete....
Hanisha Koneru [Wed, 7 Mar 2018 22:17:10 +0000 (14:17 -0800)] 
HDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete. Contributed by lufei.

11 months agoHDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contri...
Sean Mackrory [Wed, 7 Mar 2018 19:11:52 +0000 (12:11 -0700)] 
HDFS-13176. WebHdfs file path gets truncated when having semicolon (;) inside. Contributed by Zsolt Venczel.

11 months agoYARN-7626. Allow regular expression matching in container-executor.cfg for devices...
Wangda Tan [Wed, 7 Mar 2018 18:41:12 +0000 (10:41 -0800)] 
YARN-7626. Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount. (Zian Chen via wangda)

Change-Id: If461277d4557922ab7e4dce9dd8dc5d0d5f22710
(cherry picked from commit 88f9138e12d2d5a1bd13f0915acef93037c1d086)

11 months agoYARN-7891. LogAggregationIndexedFileController should support read from HAR file...
Wangda Tan [Wed, 7 Mar 2018 18:40:31 +0000 (10:40 -0800)] 
YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)

Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e

11 months agoHDFS-13222. Update getBlocks method to take minBlockSize in RPC calls. Contributed...
Tsz-Wo Nicholas Sze [Wed, 7 Mar 2018 19:27:53 +0000 (11:27 -0800)] 
HDFS-13222. Update getBlocks method to take minBlockSize in RPC calls.  Contributed by Bharat Viswanadham

11 months agoHADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled.
Steve Loughran [Wed, 7 Mar 2018 18:51:10 +0000 (18:51 +0000)] 
HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled.
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)

11 months agoYARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan
Jason Lowe [Wed, 7 Mar 2018 15:20:34 +0000 (09:20 -0600)] 
YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan

11 months agoHDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun...
Yiqun Lin [Wed, 7 Mar 2018 07:26:38 +0000 (15:26 +0800)] 
HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin.

11 months agoHDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha...
Xiaoyu Yao [Wed, 7 Mar 2018 00:44:20 +0000 (16:44 -0800)] 
HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru.

11 months agoHDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in...
Tsz-Wo Nicholas Sze [Tue, 6 Mar 2018 21:35:03 +0000 (13:35 -0800)] 
HDFS-13227. Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList.  Contributed by Shashikant Banerjee

11 months agoHDFS-13223. Reduce DiffListBySkipList memory usage. Contributed by Shashikant Banerjee
Tsz-Wo Nicholas Sze [Tue, 6 Mar 2018 20:23:03 +0000 (12:23 -0800)] 
HDFS-13223. Reduce DiffListBySkipList memory usage.  Contributed by Shashikant Banerjee

11 months agoHDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Steph...
Xiao Chen [Tue, 6 Mar 2018 17:56:36 +0000 (09:56 -0800)] 
HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell.

11 months agoHDFS-13188. Disk Balancer: Support multiple block pools during block move. Contribute...
Inigo Goiri [Tue, 6 Mar 2018 17:09:32 +0000 (09:09 -0800)] 
HDFS-13188. Disk Balancer: Support multiple block pools during block move. Contributed by Bharat Viswanadham.

11 months agoHADOOP-15291. TestMiniKdc fails on Java 9
Takanobu Asanuma [Tue, 6 Mar 2018 14:01:33 +0000 (06:01 -0800)] 
HADOOP-15291. TestMiniKdc fails on Java 9

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHADOOP-15289. FileStatus.readFields() assertion incorrect.
Steve Loughran [Tue, 6 Mar 2018 10:05:12 +0000 (10:05 +0000)] 
HADOOP-15289. FileStatus.readFields() assertion incorrect.
Contributed by Steve Loughran.

11 months agoYARN-7346. Add a profile to allow optional compilation for ATSv2 with HBase-2.0....
Rohith Sharma K S [Tue, 6 Mar 2018 05:12:03 +0000 (10:42 +0530)] 
YARN-7346. Add a profile to allow optional compilation for ATSv2 with HBase-2.0. Contributed by Haibo Chen and Rohith.

11 months agoHADOOP-15287. JDK9 JavaDoc build fails due to one-character underscore identifiers...
Takanobu Asanuma [Tue, 6 Mar 2018 02:15:53 +0000 (18:15 -0800)] 
HADOOP-15287. JDK9 JavaDoc build fails due to one-character underscore identifiers in hadoop-yarn-common

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHADOOP-15271. Remove unicode multibyte characters from JavaDoc
Takanobu Asanuma [Mon, 5 Mar 2018 23:07:58 +0000 (15:07 -0800)] 
HADOOP-15271. Remove unicode multibyte characters from JavaDoc

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoYARN-7736. Fix itemization in YARN federation document
Sen Zhao [Mon, 5 Mar 2018 19:22:09 +0000 (11:22 -0800)] 
YARN-7736. Fix itemization in YARN federation document

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoYARN-7972. Support inter-app placement constraints for allocation tags by application...
Arun Suresh [Mon, 5 Mar 2018 19:24:17 +0000 (11:24 -0800)] 
YARN-7972. Support inter-app placement constraints for allocation tags by application ID. (Weiwei Yang via asuresh)

11 months agoHADOOP-15288. TestSwiftFileSystemBlockLocation doesn't compile.
Steve Loughran [Mon, 5 Mar 2018 18:12:38 +0000 (18:12 +0000)] 
HADOOP-15288. TestSwiftFileSystemBlockLocation doesn't compile.
Contributed by Steve Loughran.

11 months agoYARN-7915. Trusted image log message repeated multiple times. Contributed by Shane...
Billie Rinaldi [Mon, 5 Mar 2018 17:08:44 +0000 (09:08 -0800)] 
YARN-7915. Trusted image log message repeated multiple times. Contributed by Shane Kumpf

11 months agoHADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate...
Steve Loughran [Mon, 5 Mar 2018 14:06:20 +0000 (14:06 +0000)] 
HADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate exceptions.
Contributed by Aaron Fabbri.

11 months agoYARN-7995. Remove unnecessary boxings and unboxings from PlacementConstraintParser...
Weiwei Yang [Mon, 5 Mar 2018 03:49:43 +0000 (11:49 +0800)] 
YARN-7995. Remove unnecessary boxings and unboxings from PlacementConstraintParser.java. Contributed by Sen Zhao.

11 months agoHADOOP-15282. HADOOP-15235 broke TestHttpFSServerWebServer
Robert Kanter [Sun, 4 Mar 2018 15:33:47 +0000 (07:33 -0800)] 
HADOOP-15282. HADOOP-15235 broke TestHttpFSServerWebServer

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHADOOP-15286. Remove unused imports from TestKMSWithZK.java
Ajay Kumar [Sun, 4 Mar 2018 15:12:52 +0000 (07:12 -0800)] 
HADOOP-15286. Remove unused imports from TestKMSWithZK.java

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contri...
Arpit Agarwal [Thu, 1 Mar 2018 19:09:43 +0000 (11:09 -0800)] 
HDFS-13178. Disk Balancer: Add a force option to DiskBalancer Execute command. Contributed by Bharat Viswanadham.

11 months agoHDFS-13173. Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store...
Tsz-Wo Nicholas Sze [Sat, 3 Mar 2018 01:46:50 +0000 (17:46 -0800)] 
HDFS-13173.  Replace ArrayList with DirectoryDiffList(SnapshotSkipList) to store DirectoryDiffs.  Contributed by Shashikant Banerjee

11 months agoHADOOP-15252. Checkstyle version is not compatible with IDEA's checkstyle plugin
Andras Bokor [Sat, 3 Mar 2018 01:08:17 +0000 (17:08 -0800)] 
HADOOP-15252. Checkstyle version is not compatible with IDEA's checkstyle plugin

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by...
Xiao Chen [Sat, 3 Mar 2018 00:45:07 +0000 (16:45 -0800)] 
HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen.

11 months agoMAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun
Gergely Novák [Sat, 3 Mar 2018 00:15:04 +0000 (16:15 -0800)] 
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHADOOP-15261. Upgrade commons-io from 2.4 to 2.5. Contributed by PandaMonkey.
Akira Ajisaka [Fri, 2 Mar 2018 23:47:43 +0000 (15:47 -0800)] 
HADOOP-15261. Upgrade commons-io from 2.4 to 2.5. Contributed by PandaMonkey.

11 months agoHDFS-13171. Handle Deletion of nodes in SnasphotSkipList. Contributed by Shashikant...
Tsz-Wo Nicholas Sze [Fri, 2 Mar 2018 18:51:05 +0000 (10:51 -0800)] 
HDFS-13171. Handle Deletion of nodes in SnasphotSkipList.  Contributed by Shashikant Banerjee

11 months agoHDFS-1686. Federation: Add more Balancer tests with federation setting. Contributed...
Tsz-Wo Nicholas Sze [Fri, 2 Mar 2018 18:42:39 +0000 (10:42 -0800)] 
HDFS-1686. Federation: Add more Balancer tests with federation setting.  Contributed by Bharat Viswanadham

11 months agoYARN-7511. NPE in ContainerLocalizer when localization failed for running container...
Jason Lowe [Fri, 2 Mar 2018 15:59:46 +0000 (09:59 -0600)] 
YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang

11 months agoHDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi.
Yiqun Lin [Fri, 2 Mar 2018 03:49:12 +0000 (11:49 +0800)] 
HDFS-13210. Fix the typo in MiniDFSCluster class. Contributed by fang zhenyi.

11 months agoHADOOP-14652. Update metrics-core version to 3.2.4. (rchiang)
Ray Chiang [Thu, 1 Mar 2018 23:34:54 +0000 (15:34 -0800)] 
HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang)

11 months agoHADOOP-15279. increase maven heap size recommendations
Allen Wittenauer [Thu, 1 Mar 2018 20:53:19 +0000 (12:53 -0800)] 
HADOOP-15279. increase maven heap size recommendations

Signed-off-by: Jason Lowe <jlowe@apache.org>
Signed-off-by: Chris Douglas <cdouglas@apache.org>
11 months agoHADOOP-15274. Move hadoop-openstack to slf4j
fang zhenyi [Thu, 1 Mar 2018 23:08:21 +0000 (15:08 -0800)] 
HADOOP-15274. Move hadoop-openstack to slf4j

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange. Contributed by Shashi...
Tsz-Wo Nicholas Sze [Thu, 1 Mar 2018 22:12:15 +0000 (14:12 -0800)] 
HDFS-13211. Fix a bug in DirectoryDiffList.getMinListForRange.  Contributed by Shashikant Banerjee

11 months agoYARN-7637. GPU volume creation command fails when work preserving is disabled at...
Sunil G [Thu, 1 Mar 2018 17:09:32 +0000 (22:39 +0530)] 
YARN-7637. GPU volume creation command fails when work preserving is disabled at NM. Contributed by Zian Chen.

11 months agoYARN-5028. RMStateStore should trim down app state for completed applications(addendu...
Yufei Gu [Thu, 1 Mar 2018 16:38:53 +0000 (08:38 -0800)] 
YARN-5028. RMStateStore should trim down app state for completed applications(addendum). Contributed by Rohith Sharma K S.

11 months agoYARN-7958. ServiceMaster should only wait for recovery of containers with id that...
Billie Rinaldi [Thu, 1 Mar 2018 01:14:16 +0000 (17:14 -0800)] 
YARN-7958. ServiceMaster should only wait for recovery of containers with id that match the current application id. Contributed by Chandni Singh

11 months agoHADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry
Nanda kumar [Thu, 1 Mar 2018 10:02:24 +0000 (19:02 +0900)] 
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
11 months agoHDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun.
Yiqun Lin [Thu, 1 Mar 2018 09:51:43 +0000 (17:51 +0800)] 
HDFS-13202. Fix the outdated javadocs in HAUtil. Contributed by Chao Sun.

11 months agoRevert "yarn-7346.07.patch"
Haibo Chen [Thu, 1 Mar 2018 05:11:36 +0000 (21:11 -0800)] 
Revert "yarn-7346.07.patch"

This reverts commit 5e37ca5bb49f945e27f49a413d08baab562dfa9c.

11 months agoYARN-7835. Race condition in NM while publishing events if second attempt is launched...
Haibo Chen [Thu, 1 Mar 2018 05:06:42 +0000 (21:06 -0800)] 
YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen)

11 months agoyarn-7346.07.patch
Haibo Chen [Thu, 22 Feb 2018 04:59:41 +0000 (20:59 -0800)] 
yarn-7346.07.patch

11 months agoMAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setEras...
Weiwei Yang [Thu, 1 Mar 2018 02:18:53 +0000 (10:18 +0800)] 
MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang.

11 months agoYARN-7985. Removed duplicate service name validation in Yarn Service.
Eric Yang [Thu, 1 Mar 2018 01:48:55 +0000 (20:48 -0500)] 
YARN-7985.  Removed duplicate service name validation in Yarn Service.
            Contributed by Chandni Singh

11 months agoYARN-7955. Improve result of calling stop on an already stopped service. Contributed...
Billie Rinaldi [Wed, 28 Feb 2018 23:01:56 +0000 (15:01 -0800)] 
YARN-7955. Improve result of calling stop on an already stopped service. Contributed by Gour Saha

11 months agoHDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs...
Tsz-Wo Nicholas Sze [Wed, 28 Feb 2018 21:42:58 +0000 (13:42 -0800)] 
HDFS-13102. Implement SnapshotSkipList class to store Multi level DirectoryDiffs.  Contributed by Shashikant Banerjee

11 months agoYARN-7954. Set component status to STOPPED when YARN service is stopped. Contributed...
Billie Rinaldi [Wed, 28 Feb 2018 19:57:24 +0000 (11:57 -0800)] 
YARN-7954. Set component status to STOPPED when YARN service is stopped. Contributed by Gour Saha

11 months agoHDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contrib...
Xiaoyu Yao [Wed, 28 Feb 2018 17:44:39 +0000 (09:44 -0800)] 
HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar.

11 months agoHDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path...
Xiaoyu Yao [Wed, 28 Feb 2018 03:50:00 +0000 (19:50 -0800)] 
HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru.

11 months agoHADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add...
Steve Loughran [Wed, 28 Feb 2018 14:42:25 +0000 (14:42 +0000)] 
HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake.
Contributed by Steve Loughran

11 months agoHADOOP-15264. AWS "shaded" SDK 1.271 is pulling in netty 4.1.17.
Steve Loughran [Wed, 28 Feb 2018 13:11:07 +0000 (13:11 +0000)] 
HADOOP-15264. AWS "shaded" SDK 1.271 is pulling in netty 4.1.17.
Contributed by Steve Loughran.

(cherry picked from commit b580196a16dcafe26d6069267a5e2242917aad84)

11 months agoYARN-7929. Support to set container execution type in SLS. (Jiandan Yang via Weiwei...
Weiwei Yang [Wed, 28 Feb 2018 09:57:28 +0000 (17:57 +0800)] 
YARN-7929. Support to set container execution type in SLS. (Jiandan Yang via Weiwei Yang)

11 months agoHDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang.
Yiqun Lin [Wed, 28 Feb 2018 02:17:43 +0000 (10:17 +0800)] 
HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang.

11 months agoHDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by...
Inigo Goiri [Wed, 28 Feb 2018 01:39:02 +0000 (17:39 -0800)] 
HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong.

11 months agoHDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDi...
Tsz-Wo Nicholas Sze [Tue, 27 Feb 2018 23:28:41 +0000 (15:28 -0800)] 
HDFS-13143. SnapshotDiff - snapshotDiffReport might be inconsistent if the snapshotDiff calculation happens between a snapshot and the current tree.  Contributed by Shashikant Banerjee

11 months agoYARN-7446. Remove --user flag when running privileged mode docker container. Contribu...
Billie Rinaldi [Tue, 27 Feb 2018 22:33:03 +0000 (14:33 -0800)] 
YARN-7446. Remove --user flag when running privileged mode docker container. Contributed by Eric Yang

11 months agoYARN-7959. Add .vm extension to PlacementConstraints.md to ensure proper filtering...
Wangda Tan [Tue, 27 Feb 2018 19:15:17 +0000 (11:15 -0800)] 
YARN-7959. Add .vm extension to PlacementConstraints.md to ensure proper filtering. (Weiwei Yang via wangda)

Change-Id: I34ec516994a5de7accaa5ef7044e443b340bd294
(cherry picked from commit 0d516b30769471d9114214b3a518e2146ff2f25e)

11 months agoYARN-7893. Document the FPGA isolation feature. (Zhankun Tang via wangda)
Wangda Tan [Tue, 27 Feb 2018 19:14:40 +0000 (11:14 -0800)] 
YARN-7893. Document the FPGA isolation feature. (Zhankun Tang via wangda)

Change-Id: I8723320377a4bd0a7db168670d183ef5543caa67
(cherry picked from commit c2af2f16718bd1d38ceb05e354f34d7bef89127d)

11 months agoMAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by...
Arpit Agarwal [Tue, 27 Feb 2018 18:27:18 +0000 (10:27 -0800)] 
MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham.

11 months agoHDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed...
Arpit Agarwal [Tue, 27 Feb 2018 18:18:07 +0000 (10:18 -0800)] 
HDFS-13181. DiskBalancer: Add an configuration for valid plan hours . Contributed by Bharat Viswanadham.

11 months agoHADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions...
Arpit Agarwal [Tue, 27 Feb 2018 16:48:13 +0000 (08:48 -0800)] 
HADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions. Contributed by Ajay Kumar.

11 months agoHDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call...
Inigo Goiri [Tue, 27 Feb 2018 16:53:00 +0000 (08:53 -0800)] 
HDFS-13192. Change the code order in getFileEncryptionInfo to avoid unnecessary call of assignment. Contributed by LiXin Ge.

11 months agoHDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA...
Inigo Goiri [Tue, 27 Feb 2018 16:48:03 +0000 (08:48 -0800)] 
HDFS-13193. Various Improvements for BlockTokenSecretManager. Contributed by BELUGA BEHR.

11 months agoHADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with Strin...
Arpit Agarwal [Tue, 27 Feb 2018 15:38:29 +0000 (07:38 -0800)] 
HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar.

11 months agoHDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by...
Yiqun Lin [Tue, 27 Feb 2018 02:48:52 +0000 (10:48 +0800)] 
HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin.

11 months agoHDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled...
Chao Sun [Mon, 26 Feb 2018 23:37:27 +0000 (15:37 -0800)] 
HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun.

11 months agoYARN-7963. Updated MockServiceAM unit test to prevent test hang.
Eric Yang [Mon, 26 Feb 2018 23:49:01 +0000 (18:49 -0500)] 
YARN-7963.  Updated MockServiceAM unit test to prevent test hang.
             Contributed by Chandni Singh

11 months agoHDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo...
Wei Yan [Mon, 26 Feb 2018 23:13:41 +0000 (15:13 -0800)] 
HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan.

11 months agoMAPREDUCE-7010. Make Job History File Permissions configurable. Contributed by Gergel...
Billie Rinaldi [Mon, 26 Feb 2018 22:32:46 +0000 (14:32 -0800)] 
MAPREDUCE-7010. Make Job History File Permissions configurable. Contributed by Gergely Novák

11 months agoHADOOP-15265. Exclude json-smart explicitly in hadoop-auth avoid being pulled in...
Arpit Agarwal [Mon, 26 Feb 2018 21:56:53 +0000 (13:56 -0800)] 
HADOOP-15265. Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively. Contributed by Nishant Bangarwa.

11 months agoHDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning...
Arpit Agarwal [Sat, 24 Feb 2018 22:52:35 +0000 (14:52 -0800)] 
HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula.

11 months agoYARN-7921. Transform a PlacementConstraint to a string expression. Contributed by...
Konstantinos Karanasos [Mon, 26 Feb 2018 20:15:16 +0000 (12:15 -0800)] 
YARN-7921. Transform a PlacementConstraint to a string expression. Contributed by Weiwei Yang.

11 months agoHDFS-12070. Failed block recovery leaves files open indefinitely and at risk for...
Kihwal Lee [Mon, 26 Feb 2018 16:28:04 +0000 (10:28 -0600)] 
HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee.

11 months agoHADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.
Arpit Agarwal [Sat, 24 Feb 2018 22:41:55 +0000 (14:41 -0800)] 
HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.

11 months agoHDFS-12865. RequestHedgingProxyProvider should handle case when none of the proxies...
Arpit Agarwal [Sat, 24 Feb 2018 22:25:56 +0000 (14:25 -0800)] 
HDFS-12865. RequestHedgingProxyProvider should handle case when none of the proxies are available. Contributed by Mukul Kumar Singh.

11 months agoHDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain.
Xiaoyu Yao [Sat, 24 Feb 2018 00:56:29 +0000 (16:56 -0800)] 
HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain.