Jun He [Tue, 27 Nov 2018 07:34:49 +0000 (07:34 +0000)] 
BIGTOP-3106: Update links in download page
Links in download page should:
1. be links to release artifacts, not links to directories.
2. link to the asc and sha files should refer to archives only for non-current
releases
3. current release links should point to apache.org/dist
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Sun, 25 Nov 2018 02:51:09 +0000 (10:51 +0800)]
BIGTOP-3103: Update download page to comply with Apache announcement requirements
To comply with announcement requirement, the download page should contain links
to all current and archived releases along with links to KEYS, checksums, and
signatures for all releases.
Current download page only provides link to latest one.
Signed-off-by: Jun He <jun.he@linaro.org>
Jun He [Fri, 5 Oct 2018 01:26:45 +0000 (01:26 +0000)] 
Update provisioner docker images to 1.3.0
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 3 Oct 2018 12:52:33 +0000 (20:52 +0800)]
BIGTOP-3089: Update provision config files with changes in 1.3.0 release
v1.3.0 supports: centos-7, fedora-26, debian-9, ubuntu-16.04, opensuse-42.3
config files in provisioner/docker should be updated to sync with these changes
Now config files contain ARCH variable to automatically load arch-dependent
repo. And docker-hadoop.sh will load correct puppet image by detecting CPU
arch. This makes only one config file is needed for a certain distro acrossing
different architectures.
This closes #402
Change-Id: Ic5d66cb837e78908fcf19c2e55f819db28866a55
Signed-off-by: Jun He <jun.he@linaro.org>
Jun He [Wed, 3 Oct 2018 14:05:24 +0000 (22:05 +0800)]
BIGTOP-3091: Set Bigtop repo to higher priority
Bigtop repo file in deploy/puppet doesn't have "priority" field.
An example is on Fedora-26, zookeeper is 3.4.9 in distro repo, while
conflicts with 3.4.6 in Bigtop stack. Similiar to BIGTOP-2796.
Change-Id: I2b1231a2cf62a30401784780c92b31ab810fe34b
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Wed, 3 Oct 2018 13:56:48 +0000 (21:56 +0800)]
BIGTOP-3090: provisioner failed on fedora-26 when deploying jdk
The puppet recipe of jdk intends to setup cacerts link with pki/java/cacerts
while jdk is not installed. This security failure will cause later bigtop
components installation failure.
Change-Id: Ib728cdf60db9feaddb3f65643ee2134158947254
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Wed, 3 Oct 2018 12:00:02 +0000 (20:00 +0800)]
BIGTOP-3088: provisioner failed to run puppet deployment on CentOS-7
Deploy with puppet on centos-7 failed.
The root cause is puppet-stdlib (4.25) in distro is not compatible with
puppet <4. Pin stdlib to 4.12.0 should fix this problem.
Change-Id: I8ec557412b660cc51399da3c81014b47aa366d6a
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Sun, 23 Sep 2018 23:25:08 +0000 (23:25 +0000)]
Bigtop 1.3.0 release
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Sun, 23 Sep 2018 23:23:12 +0000 (23:23 +0000)]
Update bigtop site to 1.3.0
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Sun, 23 Sep 2018 23:19:27 +0000 (23:19 +0000)]
Update repo URLs to 1.3.0
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 20 Sep 2018 07:49:04 +0000 (07:49 +0000)]
BIGTOP-3083: HBase and iginite-hadoop build failed on Fedora-26 due to BIGTOP-3081
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 19 Sep 2018 23:41:52 +0000 (07:41 +0800)]
Add missing scala patch for flume
The commit is to add patch to fix flume/kafka build issue which is
missed in BIGTOP-3082.
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 19 Sep 2018 17:48:25 +0000 (01:48 +0800)]
BIGTOP-3082: Fix build failure with flume-1.8+kafka-0.10.2.2
When upgrading flume to 1.8 and kafka to 0.10.2.2, the build of flume
is failed due to API changes and scala upgrade in kafka.
This patch rebased FLUME-2662/FLUME-3026 patches to flume-1.8, and for
the scala conflicts, used Anton's patch which can be found here:
https://lists.apache.org/thread.html/%3CCAJZrK_tN49GM+vCCiB+rjLL==82U=7nswoapwWmyJbVAS5F=QA@mail.gmail.com%3E
This closes #397
Change-Id: Idf8d00c22ef60d82783056fea556e9a6f517054a
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Tue, 18 Sep 2018 06:57:19 +0000 (14:57 +0800)]
BIGTOP-3081: Update HBase/Ignite-hadoop arch info
Both HBase and ignite-hadoop introduce arch dependency during build.
So the output packages should be arch dependent. The arch info in
package files should be updated to reflect these.
This closes #395
Signed-off-by: Jun He <junhe@apache.org>
Yuqi Gu [Wed, 12 Sep 2018 08:10:47 +0000 (08:10 +0000)] 
BIGTOP-3075: Drop pig packaging
Drop pig packaging as it doesn't work with Hive-2.0+. See BIGTOP-2975
This closes #396
Change-Id: I7e6ca27979864f04e36595afbd5475e628e21b07
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Jun He [Wed, 29 Aug 2018 08:12:05 +0000 (08:12 +0000)]
BIGTOP-3074: Drop oozie packaging
Change-Id: I92cb5256a7f51a04f0d859b6a812bc8fa0dfc669
Signed-off-by: Jun He <junhe@apache.org>
Yuqi Gu [Fri, 10 Aug 2018 09:17:47 +0000 (09:17 +0000)]
BIGTOP-3062: Bump flink to 1.4.2
Bump flink to 1.4.2 for Bigtop 1.3 release
This closes #380
Change-Id: Icd527338d6663d7baa7e1a3690667f95c1e12320
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 10 Sep 2018 03:19:14 +0000 (11:19 +0800)]
BIGTOP-3076: QFS build failed on ppc64le
QFS build failed on ppc64le, this is caused by conflict definition of
gf_cpu_identify.
This closes #392
Change-Id: Ibc52b426a27c6ed27c71a0e8fabeaaa91577d000
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 10 Sep 2018 03:12:29 +0000 (11:12 +0800)]
BIGTOP-3078: ignite-shmem failed to build on ppc64le
Ignite-shmem build failed on ppc64le. This is caused by ppc64le is
not supported by its config.guess.
This closes #391
Change-Id: Ife8b1afe185e086034c1fd25fbb20906c0dc15be
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 13 Nov 2017 10:11:31 +0000 (18:11 +0800)]
BIGTOP-2101: ignite-hadoop contains an arch-dependent shared lib
ignite-shmem contains x86 binary only. A build step is added to create
arch specific library for non-x86 platforms.
This closes #358
Change-Id: I9f4f03ec61250da21c13a915dc7efd5303e95d5b
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Tue, 4 Sep 2018 03:25:14 +0000 (11:25 +0800)]
BIGTOP-3077: OpenSuse-42.3 packages install error
Update libapr1 to use distro version. Fixed packages dependency issue
on aarch64.
This closes #390
Change-Id: I043b701b4bd5b89eea7488e21ace939624528310
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 27 Aug 2018 03:22:21 +0000 (11:22 +0800)]
BIGTOP-3067: Bump Hive to 2.3.3
Bump to 2.3.3 for v1.3 release.
This closes #386
Change-Id: I0a2d0ab3d58bcac1aac79db1ebeb1d089f6f435b
Signed-off-by: Jun He <junhe@apache.org>
Yuqi Gu [Fri, 17 Aug 2018 06:08:48 +0000 (06:08 +0000)]
BIGTOP-3068: Bump Hama to 0.7.1
Bump Hama to 0.7.1 for bigtop 1.3 release
This closes #388
Change-Id: I557f36500989e70c4120b43706a19459d0cb4993
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Jun He <junhe@apache.org>
Yuqi Gu [Fri, 17 Aug 2018 06:44:20 +0000 (06:44 +0000)]
BIGTOP-3055: Bump GPDB to 5.10.0
GPDB upstream has rolled out 5.10.0 release. Let's bump to this.
This closes #389
Change-Id: I1bcc96e31c4802d4a5cb87b47218b6cd6b3d4f3e
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 15 Aug 2018 07:01:59 +0000 (15:01 +0800)]
BIGTOP-3069: Bump crunch to 0.15.0
Bump to latest 0.15.0 release.
This closes #385
Change-Id: I76921880550a06efc00d55d1cdcff69817578c8f
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 15 Aug 2018 07:07:42 +0000 (15:07 +0800)]
BIGTOP-3065: Bump Hadoop to 2.8.4
Bump to latest release on 2.8 branch
Change-Id: I58e719ef66174312fd005b0bb0d11711aee69ad0
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Yuqi Gu [Fri, 17 Aug 2018 05:12:18 +0000 (05:12 +0000)]
BIGTOP-3064: Bump HBase to 1.3.2
HBase 1.3.1 is released on Apr 7, 2017,
while has latest release on 1.3.x is 1.3.2, on Mar 19, 2018.
Bump Hbase to 1.3.2 for bigtop 1.3 release.
Change-Id: Iaca7a4918023c62b56fb6e6462cdd6b2c3a8302e
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Evans Ye <evansye@apache.org>
Yuqi Gu [Fri, 17 Aug 2018 05:59:02 +0000 (05:59 +0000)]
BIGTOP-3066: Bump tez to 0.9.1
Bump to 0.9.1 for 1.3 release.
Change-Id: I9ee682bdf17dde6f768d64538776643ae4d4dcbf
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Evans Ye <evansye@apache.org>
Evans Ye [Sat, 2 Jun 2018 08:47:46 +0000 (16:47 +0800)] 
BIGTOP-3039. Change git browse link to Github
Jun He [Wed, 15 Aug 2018 07:05:43 +0000 (15:05 +0800)]
BIGTOP-3059: Bump Ambari to 2.6.1
Bump Ambari to 2.6.1 release.
This closes #379
Change-Id: I94e17d70bd78ec1d2ff59e914894ad8b0b3c5cae
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 16 Aug 2018 02:19:42 +0000 (02:19 +0000)]
Changing version to 1.3.0
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 8 Aug 2018 04:51:49 +0000 (12:51 +0800)]
BIGTOP-3061: Fix ClassNotFound issue for crunch build
Pick patch from upstream.
This closes #378
Change-Id: I3c2a2bf717fa070b61b4c1788a52874361910dbc
Signed-off-by: Jun He <junhe@apache.org>
Kevin W Monroe [Sat, 5 May 2018 13:39:31 +0000 (13:39 +0000)] 
BIGTOP-3060: juju bundle refresh (august 2018)
Closes #377
Kevin W Monroe [Thu, 2 Aug 2018 18:25:38 +0000 (13:25 -0500)]
BIGTOP-3047: juju: nagios support for zookeeper
Closes #372
Signed-off-by: Kevin W Monroe <kevin.monroe@canonical.com>
Jun He [Tue, 24 Jul 2018 06:34:40 +0000 (14:34 +0800)]
BIGTOP-3054: Missing Spark archive caused Zeppelin build failure
Zeppelin uses Spark-2.2.1 archive defined in bigtop.bom. This archive
is moved from www.apache.org to archive.apache.org.
So update zeppelin build script to point to Apache's archive site.
Change-Id: Ib637c36d3623fd856d50122875a0e65b1cc82723
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Thu, 19 Jul 2018 02:46:19 +0000 (10:46 +0800)]
BIGTOP-3053: Tez failed to build due to bower version is deprecated
Tez building failed due to the bower version is deprecated. Tez upstream uses
a workaround to upgrade it to 1.8.4.
This closes #375
Change-Id: Ib8ffa59fb4fdfc23599164fda7cbe8af30c44df7
Signed-off-by: Jun He <junhe@apache.org>
Evans Ye [Sun, 8 Jul 2018 08:50:50 +0000 (16:50 +0800)]
BIGTOP-3052. Maven version is fixed in security verification code
Evans Ye [Mon, 2 Jul 2018 17:06:58 +0000 (01:06 +0800)]
BIGTOP-3048: Revert "BIGTOP-3001. Change uid and gid for jenkins user in bigtop-toolchain"
This reverts commit
6781b266decf905c1229bf36a36af14dc988e31c.
Artem Eluferev [Mon, 25 Jun 2018 11:43:26 +0000 (14:43 +0300)] 
BIGTOP-3031: correct script to find lasted mvn version
This closes #369
Signed-off-by: Jun He <junhe@apache.org>
Olaf Flebbe [Sat, 9 Jun 2018 17:35:05 +0000 (19:35 +0200)] 
BIGTOP-3045: Remove insecure maven repository URL from gradle config
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Jun He [Fri, 8 Jun 2018 06:26:11 +0000 (14:26 +0800)]
BIGTOP-3035: Provisioner failed because init is missed in Debian-9
In provisioner/docker/docker-compose.yml, /sbin/init is used as entrypoint.
But this cmd is missed on Debian-9, which causes smoke test/provisioner
failure.
This closes #368
Change-Id: I4c3aac74f8299b23966ea8a5c30a1513c734f889
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 7 Jun 2018 01:17:01 +0000 (09:17 +0800)]
BIGTOP-3042: HDFS TestHDFSQuota doesn't match Hadoop's definition
The smoke test case of HDFS, TestHDFSQuota:testInputValues, asserts fail
on setSpaceQuota to 0, while this doesn't match Hadoop's definition for SpaceQuota.
This produces false failure, and should be removed from testcases.
This closes #366
Change-Id: I5e18a15e9b45a1057174717a37008b3c92bab0e9
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Tue, 5 Jun 2018 04:57:59 +0000 (12:57 +0800)]
BIGTOP-3040: Solr's default configsets doesn't work with smoke test cases
The default configsets used in solr is data_driven_schema_configs, which
doesn't work with solr smoke test, TestIndexing:testSearch. The reason is
in data_driven_schema_configs "name" is treated as StrField, not TextField,
so the words will not be tokenized, and lead to unexpected query result.
sample_techproducts_configs should be used.
This closes #364
Change-Id: I170c9530ed937410dcd99c975276c30777b24045
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 7 Jun 2018 03:17:50 +0000 (11:17 +0800)]
BIGTOP-2818: Ambari downloads jdk
Ambari server uses jdk package from Hortonworks by default. This package
is not aligned with distro provided, and only x86 is supported.
Update Ambari server deploy script to use distro's JDK.
This closes #367
Change-Id: If062a9a4d689ed31276a944e68254b2fda9a264e
Signed-off-by: Jun He <jun.he@linaro.org>
Olaf Flebbe [Sat, 9 Jun 2018 17:19:33 +0000 (19:19 +0200)]
BIGTOP-3044: Fix docker build command for simple sh
Olaf Flebbe [Fri, 8 Jun 2018 19:33:19 +0000 (21:33 +0200)]
BIGTOP-3043: Hide workaround for docker defect on aarch64 in build.sh
Olaf Flebbe [Thu, 31 May 2018 21:16:09 +0000 (23:16 +0200)]
BIGTOP-3037: Download maven securely
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Jun He [Tue, 5 Jun 2018 07:09:48 +0000 (15:09 +0800)]
BIGTOP-3041: Failed to init hadoop hdfs using init-hdfs.sh
Failed to initialize hdfs using init-hdfs.sh script with 2.8.1.
The root cause is some jars are missed in CLASS_PATH
This closes #365
Change-Id: I6e3e1f42d437097ee122150aa94b41780346fb4f
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Wed, 9 May 2018 08:01:29 +0000 (16:01 +0800)]
BIGTOP-3027: Wrong Leveldbjni native binary for aarch64
Leveldbjni used in Hadoop is v1.8, which only includes x86-64 native binaries.
It needs to build from source when creating hadoop packages on aarch64, like
what ppc does.
This closes #360
Change-Id: I4682e7885d9b5ac6d448d954ddffc78256fe0733
Signed-off-by: Jun He <junhe@apache.org>
Sahil Aggarwal [Tue, 5 Jun 2018 13:56:21 +0000 (19:26 +0530)] 
BIGTOP-3038: Add rollingupgrade option in hadoop-hdfs-namenode.svc
Signed-off-by: Evans Ye <evansye@apache.org>
Olaf Flebbe [Wed, 30 May 2018 18:47:43 +0000 (20:47 +0200)]
BIGTOP-3036: Download gradle and groovy via TLS and do signature checking on ant
Thanks to brian m. carlson for reporting
Jun He [Fri, 18 May 2018 03:08:49 +0000 (11:08 +0800)]
BIGTOP-3033: Spark build on OpenSUSE is failed
Spark building on OpenSUSE is failed as pdflatex is missing.
Add package to install list.
This closes #359
Change-Id: Ib3914ab05fd37418a5aa798fa37b88cb54e4079e
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 10 May 2018 09:29:32 +0000 (17:29 +0800)]
BIGTOP-3030: Remove phantomjs dependecy in build
Upstream Ambari depends on phantomjs to build. And at present phantomjs provides
only x86* prebuilt binaries. This leads to build failure on non-x86 platforms.
As phantomjs is used for test, not runtime dep, it can be removed from build to
solve this failure.
This closes #357
Change-Id: I06b4fa48705024bb5de9a9167c337db546abbc36
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 14 May 2018 05:56:04 +0000 (13:56 +0800)]
BIGTOP-3031: Auto find latest maven dist
Similar to ant, maven dist binaries may get updated from time to time on
Apache mirrors. A better way is to auto find latest maven-3 binary release.
Used the same way in BIGTOP-3022.
This closes #356
Change-Id: I0384a656701f62bd7ef7fc40cd6ab85533ba49fc
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Thu, 3 May 2018 03:06:14 +0000 (11:06 +0800)]
BIGTOP-3024: Zeppelin build is failed after Spark is bumpped to 2.2.1
After Spark is bumpped to 2.2.1 by BIGTOP-2991, zeppelin build is failed.
The root cause is zeppelin now try to download spark-2.2.1.tgz and
spark-2.2.1-bin-without-hadoop.tgz from http://d3kbcqa49mib13.cloudfront.net.
This site currently doesn't offer spark 2.2.1 related packages.
So the download link need to be updated to apache site:
https://www.apache.org/dist/spark/
This closes #355
Change-Id: Icea4dc0ac6d65104c7a071e7b044161423dffec5
Signed-off-by: Jun He <junhe@apache.org>
José Pekkarinen [Fri, 2 Mar 2018 14:20:25 +0000 (16:20 +0200)] 
BIGTOP-3011: zookeeper autopurge options
Closes #346
Signed-off-by: Kevin W Monroe <kevin.monroe@canonical.com>
Jun He [Tue, 24 Apr 2018 06:11:34 +0000 (14:11 +0800)]
BIGTOP-3023: Bump qfs to 2.0.0
Prior to v2.0.0, qfs doesn't build with openssl-1.1.
This makes bigtop failed on debian-9.
Bump to v2.0.0 should solve this problem.
Change-Id: I4665e6de206ee928d1d8fbf21694bb92ef11752c
Signed-off-by: Jun He <jun.he@linaro.org>
Olaf Flebbe [Tue, 10 Apr 2018 18:51:04 +0000 (20:51 +0200)] 
BIGTOP-3021: Remove HUE from Bigtop
Jun He [Tue, 24 Apr 2018 04:59:30 +0000 (12:59 +0800)]
BIGTOP-2991: Bump up Spark version to 2.2.1
This closes #353
Change-Id: I5e5b47db667206bd4a810b1015793d69c1223cde
Signed-off-by: Jun He <junhe@apache.org>
Anton Chevychalov [Tue, 17 Apr 2018 11:20:39 +0000 (14:20 +0300)] 
BIGTOP-3022 Auto find of latest Ant
That patch allow to derminate latest version of Ant
available on apache dist.
There is one restriction. We query main apache mirror instead
of nearest one because we can't be sure it use Apache with same
configuration as main.
Signed-off-by: Olaf Flebbe <oflebbe@apache.org>
zhtisi [Fri, 30 Mar 2018 04:46:06 +0000 (12:46 +0800)] 
BIGTOP-3018:detect-javahome script has a typo in variable name
https://issues.apache.org/jira/browse/BIGTOP-3018
This closes #350
Signed-off-by: Jun He <jun.he@apache.org>
Yuqi Gu [Thu, 29 Mar 2018 08:10:55 +0000 (08:10 +0000)]
BIGTOP-2723:Fix asciidoctor-maven-plugin for HBase build on AArch64
This closes #200
Change-Id: Ia7318eb115027ceabeff9952588e5ec364b00144
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
Signed-off-by: Jun He <jun.he@apache.org>
Kevin W Monroe [Sat, 10 Mar 2018 00:26:52 +0000 (18:26 -0600)]
BIGTOP-3014: juju: use charm-env for shebangs
Fixes #349
Kevin W Monroe [Fri, 9 Mar 2018 19:23:25 +0000 (13:23 -0600)]
BIGTOP-3013: kafka charm: remove bigtop_version override
Fixes #347
Kevin W Monroe [Wed, 18 Oct 2017 15:59:05 +0000 (15:59 +0000)]
BIGTOP-3010: juju bundle refresh (feb 2018)
Closes #345
José Pekkarinen [Fri, 23 Feb 2018 08:36:14 +0000 (10:36 +0200)]
BIGTOP-3005: zookeeper charm: support relation changes
Closes #341
Signed-off-by: Kevin W Monroe <kevin.monroe@canonical.com>
jamesbeedy [Tue, 27 Feb 2018 17:19:06 +0000 (09:19 -0800)] 
BIGTOP-3007: hive pkg and charm opts for zookeeper
Closes #344
Signed-off-by: Kevin W Monroe <kevin.monroe@canonical.com>
Jun He [Mon, 26 Feb 2018 08:06:44 +0000 (16:06 +0800)]
BIGTOP-3003: Ant-1.9.9 tarball is removed from Apache mirrors
As from 06-Feb, ant-1.9.10 is released and distributed
to Apache mirrors. Ant-1.9.9 is removed and can only
be found at Apache's archive. So bump ant to 1.9.10.
This closes #340
Change-Id: I8175684444b08a1607ee04589f5d252dac5588e2
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Mon, 26 Feb 2018 07:28:42 +0000 (15:28 +0800)]
BIGTOP-3004: Fix HBase build failure on Debian/Fedora
Debian/Fedora uses bash-4.4.12 which treats the extra
trailing backslash as error. HBase has patch for this
and should be backported.
Change-Id: I11353ec0fcdca6e04373c0dac34e0e23aa757f30
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Akira Ajisaka [Tue, 27 Feb 2018 10:01:29 +0000 (19:01 +0900)] 
BIGTOP-3001. Change uid and gid for jenkins user in bigtop-toolchain
This closes #343
Signed-off-by: Jun He <junhe@apache.org>
Jun He [Tue, 27 Feb 2018 07:57:04 +0000 (15:57 +0800)]
BIGTOP-3006: Add Jun He to team list
Change-Id: I07e193bfad34d6fe393b02dd61270d6e0a6394bc
Signed-off-by: Jun He <junhe@apache.org>
Gregory Van Seghbroeck [Fri, 23 Feb 2018 20:07:47 +0000 (21:07 +0100)] 
BIGTOP-3002: For Kafka, it should be possible to set the broker.id config and log.dirs config.
Added 2 new parameters: broker_id, log_dirs. Both do not necessarily
need to be set. It is only when they are set , the parameters are used
in the server.properties file.
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Jun He [Tue, 14 Nov 2017 09:17:13 +0000 (17:17 +0800)]
BIGTOP-2922: Drop arch specific bigtop-* images
Use unified Dockerfile for puppet/slaves images on different architectures.
Simplified the build process, and add gradle tasks for them.
Example: ./gradlew -POS=debian-9 -Pprefix=1.2.1 bigtop-puppet
./gradlew -POS=debian-9 -Pprefix=1.2.1 bigtop-slaves
This will create proper images with same image name as existings on different
HW architectures. This eases efforts for deploy and test.
Change-Id: I183452324faf54808a4e88981d604d8fe119b51a
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Hari Krishna Dara [Sun, 4 Feb 2018 16:16:52 +0000 (21:46 +0530)] 
BIGTOP-2992: Building Pig with pre-arranged Forrest
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Hari Krishna Dara [Sun, 4 Feb 2018 16:08:42 +0000 (21:38 +0530)]
BIGTOP-2989: Support a local dir with tomcat, when download can't work
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Jun He [Thu, 1 Feb 2018 09:56:33 +0000 (17:56 +0800)]
BIGTOP-2988: Incorrect use "dnf update" in puppetize.sh
The cmd "dnf -y update" is not correctly used as the
purpose here is to "synchronize the package index
files from their sources".
For this purpose the cmd should be "dnf check-update"
Change-Id: I9ed7b7b631f5fabc00a4ef679d30a68f5c68b539
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Jun He [Thu, 19 Oct 2017 08:45:40 +0000 (16:45 +0800)]
BIGTOP-2912: Initialize dataset for solr smoke test
Change-Id: Ic55033d04f7433eaa6869b8df9c44170b1b7ec0f
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Evans Ye <evansye@apache.org>
Evans Ye [Wed, 31 Jan 2018 13:27:53 +0000 (21:27 +0800)]
BIGTOP-2984. Rat check failed after BIGTOP-2698
Evans Ye [Tue, 28 Nov 2017 16:22:14 +0000 (00:22 +0800)]
BIGTOP-2949. Add gradle task which leverage bigtop-ci/build.sh to build packages
Youngwoo Kim [Sat, 3 Feb 2018 03:53:09 +0000 (12:53 +0900)] 
BIGTOP-2990: Upgrade Phoenix version to 4.13.1-HBase-1.3
Anton Chevychalov [Fri, 8 Sep 2017 16:52:14 +0000 (19:52 +0300)]
BIGTOP-2894 Bump hbase to 1.3.1
There are a number of issues with licenses check when hbase
building with Hadoop 2.8.x
That patch bump hbase to latest release and add couple patches.
Signed-off-by: Youngwoo Kim <ywkim@apache.org>
Youngwoo Kim [Wed, 27 Dec 2017 05:24:36 +0000 (14:24 +0900)]
BIGTOP-2974: Add puppet code to deploy SparkR package
Signed-off-by: Evans Ye <evansye@apache.org>
Jun He [Fri, 5 Jan 2018 10:05:53 +0000 (18:05 +0800)]
BIGTOP-2980: Hama does not build on DEB type systems
Add source/format and modify "rules" file to comply
with Debian format "3.0 (quilt)".
Change-Id: I5aaaf89eb6f0a043bac45f4a4a610cbbdef1b91c
Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Youngwoo Kim [Thu, 4 Jan 2018 01:28:41 +0000 (10:28 +0900)]
BIGTOP-2981: Packaging SparkR is broken due to BIGTOP-2959
Olaf Flebbe [Mon, 1 Jan 2018 18:18:07 +0000 (19:18 +0100)]
BIGTOP-2979: JAVA_HOME inconsistent on non-x86 architectures
Youngwoo Kim [Sat, 30 Dec 2017 06:21:27 +0000 (15:21 +0900)]
BIGTOP-2978: Fix bigtop_toolchain for fedora: no epel-release necessary
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Olaf Flebbe [Thu, 28 Dec 2017 15:07:10 +0000 (16:07 +0100)]
BIGTOP-2976: zookeeper-rest missing dependency on lsb-base
Youngwoo Kim [Wed, 27 Dec 2017 02:44:51 +0000 (11:44 +0900)]
BIGTOP-2973: Hama packaging is broken with Hadoop 2.8+
Youngwoo Kim [Tue, 26 Dec 2017 06:17:19 +0000 (15:17 +0900)]
BIGTOP-2959: Add SPARK_DIST_CLASSPATH to include libraries for HDFS and YARN
Youngwoo Kim [Tue, 26 Dec 2017 16:45:40 +0000 (01:45 +0900)]
BIGTOP-2970: Bump Zeppelin version to 0.7.3
Youngwoo Kim [Tue, 26 Dec 2017 14:08:32 +0000 (23:08 +0900)]
BIGTOP-2969: Bump up Spark version to 2.2.0
sahil aggarwal [Tue, 14 Mar 2017 12:08:57 +0000 (17:38 +0530)]
BIGTOP-2698: Build/install protobuf from source for all platforms
Includes additional changes by Olaf Flebbe
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
Youngwoo Kim [Thu, 15 Jun 2017 01:59:21 +0000 (10:59 +0900)]
BIGTOP-2809: Support R integration for Spark
YoungWoo Kim [Sat, 7 Nov 2015 15:07:16 +0000 (00:07 +0900)]
BIGTOP-2968: Bump Hive version to 2.3.2
Youngwoo Kim [Wed, 13 Dec 2017 08:24:24 +0000 (17:24 +0900)]
BIGTOP-2811: Add R to toolchain for Spark and Zeppelin
Anton Chevychalov [Mon, 2 Oct 2017 15:55:12 +0000 (18:55 +0300)]
BIGTOP-2898 Remove h2 pig classifier in Phoenix
There is a regression in Pig relase 0.17
After PIG-4923 they drop h2 classifer.
There is only one jar now and it suports Hadoop 2
That commit also adds BIGTOP_PATCH macro to Phoenix
Signed-off-by: Youngwoo Kim <ywkim@apache.org>
Anton Chevychalov [Fri, 29 Sep 2017 16:09:10 +0000 (19:09 +0300)]
BIGTOP-2899 remove h2 pig classifier in Oozie code
There is a regression in Pig relase 0.17
After PIG-4923 they drop h2 classifer.
There is only one jar now and it suports Hadoop 2
Signed-off-by: Youngwoo Kim <ywkim@apache.org>
Anton Chevychalov [Thu, 14 Sep 2017 12:01:06 +0000 (15:01 +0300)]
BIGTOP-2895 Bump Pig to 0.17.0 with Hadoop-2.8
Bump Pig to 0.17.0.
Remove useless patches PIG-4676 and PIG-4728 it is already in 0.17 release.
Add patch PIG-5304 to build pig with Hadoop 2.8
Signed-off-by: Youngwoo Kim <ywkim@apache.org>
Anton Chevychalov [Wed, 27 Sep 2017 11:48:38 +0000 (14:48 +0300)]
BIGTOP-2893 Bump Hadoop to 2.8.1
Patches:
=======
HADOOP-12366 was ported to 2.8.1
HADOOP-11628 was removed because it was merged to 2.8
Build time issues fixed:
========================
Fuse_dfs target dir has been changed in 2.8
-------------------------------------------
cp: cannot stat 'hadoop-hdfs-project/hadoop-hdfs/target/native/main/native/fuse-dfs/fuse_dfs': No such file or directory
add package goal to mvn
----------------------
Without that goal tar is incomplete:
for dir in '${HADOOP_DIR}/{lib,}' '${HDFS_DIR}/{lib,}' '${YARN_DIR}/{lib,}' '${MAPREDUCE_DIR}/{lib,}'
'[' -e /code/build/hadoop/rpm/BUILDROOT/hadoop-2.8.1-243.el7.centos.x86_64/usr/lib/hadoop-mapreduce//hadoop-common-2.8.1.jar ']'
exit 1
error: Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)
Bad exit status from /var/tmp/rpm-tmp.5Etrah (%install)
Signed-off-by: Youngwoo Kim <ywkim@apache.org>
Youngwoo Kim [Fri, 8 Dec 2017 02:46:29 +0000 (11:46 +0900)]
BIGTOP-2957: Upgrade YCSB to 0.12.0
projects / bigtop.git / log
