falcon.git
6 years agoFALCON-2101 Disable IT tests that depend on instance status reaching … release-0.10-rc1
bvellanki [Thu, 28 Jul 2016 07:06:47 +0000 (00:06 -0700)] 
FALCON-2101 Disable IT tests that depend on instance status reaching …

…RUNNING state

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>,Peeyush B <pbishnoi@hortonworks.com>,Pallavi Rao <pallavi.rao@inmobi.com>"

Closes #247 from bvellanki/FALCON-2101

(cherry picked from commit e94dd72fad805eed25f4d035ffc90e7e71e3dffd)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2101 Disable IT tests that depend on instance state
bvellanki [Thu, 28 Jul 2016 05:39:09 +0000 (22:39 -0700)] 
FALCON-2101 Disable IT tests that depend on instance state

Unblocking release by disabling indeterministic tests. Create FALCON-2102 for a more thorough investigation and fix.

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Peeyush B <pbishnoi@hortonworks.com>"

Closes #245 from bvellanki/FALCON-2101

(cherry picked from commit f63795155dfd19844f0733837cfaab350d92cee2)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2005 Update CHANGES.0.10.md for 0.10 release
bvellanki [Tue, 26 Jul 2016 23:17:07 +0000 (16:17 -0700)] 
FALCON-2005 Update CHANGES.0.10.md for 0.10 release

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #243 from bvellanki/FALCON-2005

(cherry picked from commit c588628aa94eb7e924776127cbdcd2398d533ba1)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2100 Remove dependency on com.vividsolutions.jts
bvellanki [Tue, 26 Jul 2016 22:07:28 +0000 (15:07 -0700)] 
FALCON-2100 Remove dependency on com.vividsolutions.jts

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #242 from bvellanki/FALCON-2100

(cherry picked from commit 84857961d6f9dca9616ceb222b5e01eca63922d2)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2099 Update Installation-steps.txt and NOTICE.txt for 0.10 rel
bvellanki [Tue, 26 Jul 2016 22:05:01 +0000 (15:05 -0700)] 
FALCON-2099 Update Installation-steps.txt and NOTICE.txt for 0.10 rel

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #241 from bvellanki/FALCON-2099

(cherry picked from commit 7ec539a8c854f6cf8ad1f77f7ad84a840ef2c577)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2090 HDFS Snapshot failed with UnknownHostException in HA mode release-0.10-rc0
bvellanki [Thu, 21 Jul 2016 18:44:14 +0000 (11:44 -0700)] 
FALCON-2090 HDFS Snapshot failed with UnknownHostException in HA mode

…uling in HA Mode

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Peeyush B <pbishnoi@hortonworks.com>"

Closes #237 from bvellanki/FALCON-2090

(cherry picked from commit 3ed804e92d1434e42a057dc68a6faa6d2fb248ce)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2081 ExtensionManagerIT fails occassionally
bvellanki [Wed, 20 Jul 2016 19:30:19 +0000 (12:30 -0700)] 
FALCON-2081 ExtensionManagerIT fails occassionally

Fixed a rat-check error as part of this Jira as it was blocking testing and it was only 1 line change

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #235 from bvellanki/FALCON-2081

(cherry picked from commit 40d943637067b04747b337e714f1588567fa4751)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2063 Add change log for each release
Ajay Yadava [Wed, 20 Jul 2016 16:58:08 +0000 (09:58 -0700)] 
FALCON-2063 Add change log for each release

Author: Ajay Yadava <ajayyadava@apache.org>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #217 from ajayyadava/2063

(cherry picked from commit 42f47b277bca48f814f785892fd1e3b9343daa98)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2084 HCatReplicationTest are failing in secure mode
Venkat Ranganathan [Wed, 20 Jul 2016 00:05:57 +0000 (17:05 -0700)] 
FALCON-2084 HCatReplicationTest are failing in secure mode

Removed the dependency on job-xml whose priority has changed because of some backward incompatible changes in Oozie - this patch will work with 4.2.0 and upcoming 4.3.0

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Ying Zheng <yzheng@hortonworks.com>"

Closes #231 from vrangan/FALCON-2084

(cherry picked from commit ec3e49e393c45ca580795e47fb085c8e58a8f789)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2075 Falcon HiveDR tasks do not report progress and can get killed
Venkat Ranganathan [Tue, 19 Jul 2016 20:23:20 +0000 (13:23 -0700)] 
FALCON-2075 Falcon HiveDR tasks do not report progress and can get killed

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Praveen Adlakha <adlakha.praveen@gmail.com>, Balu Vellanki <balu@apache.org>"

Closes #230 from vrangan/FALCON-2075

(cherry picked from commit ec4a273a8776ebdd1c50e062cad46c981b1d0122)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2071 Falcon Spark SQL failing with Yarn Client Mode
peeyush b [Tue, 19 Jul 2016 20:21:51 +0000 (13:21 -0700)] 
FALCON-2071 Falcon Spark SQL failing with Yarn Client Mode

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Praveen Adlakha <adlakha.praveen@gmail.com>, Balu Vellanki <balu@apache.org>, Venkat Ranganathan  <venkat@hortonworks.com>"

Closes #220 from peeyushb/FALCON-2071

(cherry picked from commit 73339264d6d8cd627aee0be53007a7bd4e4956ac)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2088 Entity submission fails with EntityNotRegisteredException in distributed...
Praveen Adlakha [Tue, 19 Jul 2016 18:31:37 +0000 (11:31 -0700)] 
FALCON-2088 Entity submission fails with EntityNotRegisteredException in distributed mode

… in distributed mode

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Pallavi Rao <pallavi@apache.org>"

Closes #232 from PraveenAdlakha/2088

(cherry picked from commit 9a4c18c7d380334756df84f89f57dabc42437008)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2060 Retry does not happen if instance timedout
Pallavi Rao [Fri, 15 Jul 2016 08:56:19 +0000 (14:26 +0530)] 
FALCON-2060 Retry does not happen if instance timedout

These got broken when we moved from wf rerun to coord rerun.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @peeyushb, @sandeepSamudrala

Closes #228 from pallavi-rao/2060

(cherry picked from commit 0da2074040b95c64b49ab6fb65e57644bb022e55)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2067 In instance status list, -allAttempts does not show all runs
Pallavi Rao [Thu, 14 Jul 2016 11:11:51 +0000 (16:41 +0530)] 
FALCON-2067 In instance status list, -allAttempts does not show all runs

After the fix:
`$ bin/falcon instance -status -type process -name oozie-mr-process -start 2013-11-15T00:00Z -allAttempts
Consolidated Status: SUCCEEDED
Instances:
Instance Cluster SourceCluster Status Start End Details Log
-----------------------------------------------------------------------------------------------
2013-11-15T00:05Z local - READY - - - -
2013-11-15T00:05Z local - KILLED 2016-07-14T05:53Z 2016-07-14T05:53Z - http://IM1738M1:11000/oozie?job=0000006-160713101246755-oozie-oozi-W
actions:
 user-action ERROR null
2013-11-15T00:05Z local - KILLED 2016-07-14T05:48Z 2016-07-14T05:51Z - http://IM1738M1:11000/oozie?job=0000003-160713101246755-oozie-oozi-W
actions:
 user-action ERROR null
 failed-post-processing KILLED http://localhost:8088/proxy/application_1468321868387_0012/
2013-11-15T00:10Z local - READY - - - -
2013-11-15T00:10Z local - KILLED 2016-07-14T05:45Z 2016-07-14T05:48Z - http://IM1738M1:11000/oozie?job=0000004-160713101246755-oozie-oozi-W
actions:
 user-action ERROR null
 failed-post-processing KILLED http://localhost:8088/proxy/application_1468321868387_0011/
2013-11-15T00:15Z local - KILLED - - - -
2013-11-15T00:20Z local - KILLED - - - -
2013-11-15T00:25Z local - KILLED - - - -
2013-11-15T00:30Z local - KILLED - - - -
2013-11-15T00:35Z local - KILLED - - - -`

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @peeyushb, @PraveenAdlakha

Closes #226 from pallavi-rao/2067 and squashes the following commits:

06980e6 [Pallavi Rao] FALCON-2067 Updated patch to return Oozie workflow id, rather than coord id
39aa1e5 [Pallavi Rao] FALCON-2067 In instance status list, -allAttempts does not show all runs

(cherry picked from commit 6610f36b131f53b7bb89c40a9f91b2a57bd585af)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-1986 logFile retrieved when getting process instance status should be a link...
Pallavi Rao [Wed, 8 Jun 2016 18:12:18 +0000 (11:12 -0700)] 
FALCON-1986 logFile retrieved when getting process instance status should be a link that points to logs

…ould be a link that points to logs

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: "Sandeep<sandysmdl@gmail.com>, Peeyush<peeyushb@apache.org>, Balu <balu@apache.org>"

Closes #177 from pallavi-rao/1986

(cherry picked from commit 32be559a061e209b0fe183cf825be21ae4193136)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-1749 Instance status does not show instances if entity is dele…
Praveen Adlakha [Thu, 14 Jul 2016 09:41:44 +0000 (15:11 +0530)] 
FALCON-1749 Instance status does not show instances if entity is dele…

…ted from one of the colos

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: @bvellanki, @pallavi-rao

Closes #222 from PraveenAdlakha/1749_V2 and squashes the following commits:

e476a85 [Praveen Adlakha] Pallavi's comment addressed
84e389f [Praveen Adlakha] Comments added for future reference
dbc7e1c [Praveen Adlakha] FALCON-1749 Instance status does not show instances if entity is deleted from one of the colos

(cherry picked from commit 37848ed653dbebcee5cbaec546ccc7a9d1fdb4fe)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2076 Server fails to start since extension.store.uri is empty in startup.prope...
bvellanki [Thu, 14 Jul 2016 09:38:06 +0000 (15:08 +0530)] 
FALCON-2076 Server fails to start since extension.store.uri is empty in startup.properties

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: @pallavi-rao, @sandeepSamudrala

Closes #225 from bvellanki/FALCON-2076

(cherry picked from commit afb07cc2c7b842df928597a9e54cf05e679dab82)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-1801 [ UPDATE CHANGES.TXT TO MARK 0.9 AS RELEASED]
Ajay Yadava [Tue, 12 Jul 2016 22:17:45 +0000 (15:17 -0700)] 
FALCON-1801 [ UPDATE CHANGES.TXT TO MARK 0.9 AS RELEASED]

Author: Ajay Yadava <ajaynsit@gmail.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #215 from ajayyadava/1801

(cherry picked from commit e46e5c4c641fd5e35eb1fbdacbeb04d116f3a6e7)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2065 Falcon CLI failing with missing KerberosAuthenticator class
peeyush b [Fri, 8 Jul 2016 16:11:54 +0000 (09:11 -0700)] 
FALCON-2065 Falcon CLI failing with missing KerberosAuthenticator class

Pull request for Falcon 0.10.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Balu <balu@apache.org>"

Closes #219 from peeyushb/FALCON-2065_0.10

6 years agoFALCON-2061 Falcon CLI shows hadoop classpath loading info in the console
bvellanki [Thu, 7 Jul 2016 18:33:41 +0000 (11:33 -0700)] 
FALCON-2061 Falcon CLI shows hadoop classpath loading info in the console

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>, Venkatesan Ramachandran <Me.venkatr@gmail.com>"

Closes #213 from bvellanki/FALCON-2061

(cherry picked from commit 91f7c817c4c82399a5e7e8eef8e4d60450b92e4a)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2050 Configure jetty parent classloader to be prioritized over webapp classloader
Venkat Ranganathan [Fri, 1 Jul 2016 18:09:25 +0000 (11:09 -0700)] 
FALCON-2050 Configure jetty parent classloader to be prioritized over webapp classloader

Enabled the parent classloader to have priority over the webapp classloader

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Sandeep Samudrala <sandysmdl@gmail.com>"

Closes #211 from vrangan/0.10

6 years agoFALCON-2056 HiveDR doesn't work with multiple users
Sowmya Ramesh [Fri, 1 Jul 2016 17:45:28 +0000 (10:45 -0700)] 
FALCON-2056 HiveDR doesn't work with multiple users

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Peeyush B <peeyushb@apache.org>, Balu Vellanki <balu@apache.org>, Ying Zheng <yzheng@hortonworks.com>"

Closes #207 from sowmyaramesh/FALCON-2056

(cherry picked from commit 9018ed3458d4d1def5c0643fa0e45a61fc5d82f7)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2048 Cluster submission failed in yarn-cluster mode
peeyush b [Fri, 1 Jul 2016 17:31:15 +0000 (10:31 -0700)] 
FALCON-2048 Cluster submission failed in yarn-cluster mode

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>"

Closes #210 from peeyushb/FALCON-2048

(cherry picked from commit 57fa2ff125602706b26b6d168a84b828c490ff74)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1938 Add support to execute Spark SQL process
peeyush b [Fri, 1 Jul 2016 16:56:21 +0000 (09:56 -0700)] 
FALCON-1938 Add support to execute Spark SQL process

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Venkat Ranganathan  <venkat@hortonworks.com>"

Closes #188 from peeyushb/FALCON-1938

(cherry picked from commit c12c999b4cc89bd3fbe24873567f441eed02a4ef)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2058 s3 tests with dummy url no longer compatible with latest HDFS
yzheng-hortonworks [Fri, 1 Jul 2016 16:52:00 +0000 (09:52 -0700)] 
FALCON-2058 s3 tests with dummy url no longer compatible with latest HDFS

With latest HDFS 2.7.2 or above, falcon's validation with dummy s3 url would fail. Therefore disabling this s3 replication UT. Valid s3 case is already covered in regression tests.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Peeyush B <peeyushb@apache.org>, Balu Vellanki <balu@apache.org>"

Closes #206 from yzheng-hortonworks/FALCON-2058 and squashes the following commits:

10f6006 [yzheng-hortonworks] review by peeyush
f6c825c [yzheng-hortonworks] FALCON-2058 s3 tests with dummy url no longer compatible with latest HDFS

(cherry picked from commit bb23ccf08cdc0a0a7458218bbb559947ac6e1d2d)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2037 HiveDR Extension tests are failed in Secure mode with clusterForJobNNKerb...
Sowmya Ramesh [Fri, 1 Jul 2016 16:46:34 +0000 (09:46 -0700)] 
FALCON-2037 HiveDR Extension tests are failed in Secure mode with clusterForJobNNKerberosPrincipal not found

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Peeyush B <peeyushb@apache.org>, Balu Vellanki <balu@apache.org>"

Closes #209 from sowmyaramesh/FALCON-2037

(cherry picked from commit 679c9f51dda6a045b900c62b59cb2c8b0a6a0ac8)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2046 HDFS Replication failing in secure Mode
Sowmya Ramesh [Fri, 1 Jul 2016 16:45:01 +0000 (09:45 -0700)] 
FALCON-2046 HDFS Replication failing in secure Mode

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Peeyush B <peeyushb@apache.org>, Balu Vellanki <balu@apache.org>"

Closes #208 from sowmyaramesh/FALCON-2046

(cherry picked from commit 31ab502391cd1a75cbad28259d769088273ac0f4)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2049 Feed Replication with Empty Directories are failing
bvellanki [Fri, 1 Jul 2016 16:42:31 +0000 (09:42 -0700)] 
FALCON-2049 Feed Replication with Empty Directories are failing

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Ying Zheng <yzheng@hortonworks.com>, Peeyush B <peeyushb@apache.org>, Pallavi Rao <pallavi.rao@inmobi.com>"

Closes #204 from bvellanki/FALCON-2049

(cherry picked from commit b135f28f314b1ec0a89c2e1c21b033e48e7db602)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2057 HiveDR not working with multiple users and same DB
bvellanki [Fri, 1 Jul 2016 00:21:06 +0000 (17:21 -0700)] 
FALCON-2057 HiveDR not working with multiple users and same DB

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "yzheng-hortonworks <yzheng@hortonworks.com>, Sowmya Ramesh <sramesh@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #203 from bvellanki/FALCON-2057

(cherry picked from commit d0bc188601566dbf76555a115c7ea9c68ca13909)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2051 PostProcessing needs to send JMS message so that REPL metrics can be...
Venkatesan Ramachandran [Thu, 30 Jun 2016 09:59:58 +0000 (15:29 +0530)] 
FALCON-2051 PostProcessing needs to send JMS message so that REPL metrics can be added to the GraphDB

Basically, reverted the FALCON-1926 changes.
We will attempt to solve it properly by not sending JMS message from Falcon PostProcessing after 0.10 release.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: Venkat <n.r.v@gmail.com>, Pallavi Rao <pallavi.rao@inmobi.com>, Praveen Adlakha <adlakha.praveen@gmail.com>, Peeyush<peeyushb@apache.org>

Closes #201 from vramachan/FALCON-2051.PostProcessingNotInvoked.0.10

6 years agoFALCON-2045 Enhance document on registry point in cluster specifiction for Hive HA...
yzheng-hortonworks [Fri, 24 Jun 2016 22:23:17 +0000 (15:23 -0700)] 
FALCON-2045 Enhance document on registry point in cluster specifiction for Hive HA mode

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #197 from yzheng-hortonworks/FALCON-2045

(cherry picked from commit e8d1de35d60d32d499530412fc4592abaf16f544)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1931 : Adding multiCluster tag for Multiple Cluster scenarios
Murali Ramasami [Thu, 23 Jun 2016 20:48:38 +0000 (13:48 -0700)] 
FALCON-1931 : Adding multiCluster tag for Multiple Cluster scenarios

In the regression test suite, there are lot of places multiCluster annotation has been defined to denote it uses multiple cluster. It is very useful annotation since we require additional cluster and setup to test the test case . Some of the major test cases such as feed replication and Hive DR are missing this annotation.

Author: Murali Ramasami <mramasami@Murali-Ramasami-MBP.local>
Author: Murali Ramasami <mramasami@hortonworks.com>

Reviewers: "Raghav Gautam <raghav@apache.org>"

Closes #123 from muraliramasami/multiClusterTag

(cherry picked from commit 7dab15f0036c56c06a29d36a5df0527823a930f8)
Signed-off-by: Sowmya Ramesh <sramesh@hortonworks.com>
6 years agoFALCON-2035 Entity list operation without type parameter doesn't work when authorizat...
yzheng-hortonworks [Tue, 21 Jun 2016 06:24:09 +0000 (11:54 +0530)] 
FALCON-2035 Entity list operation without type parameter doesn't work when authorization is enabled

Test entity list operation without type parameter works when authorization is enabled.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: @pallavi-rao, @bvellanki

Closes #189 from yzheng-hortonworks/FALCON-2035 and squashes the following commits:

0cbd7a4 [yzheng-hortonworks] review by pallavi
4f7c4a4 [yzheng-hortonworks] review by balu
7aa034b [yzheng-hortonworks] FALCON-2035 Entity list operation without type parameter doesn't work when authorization is enabled

(cherry picked from commit aaf33261f32ba87d95c268e09a593f9ebae1f46b)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2038 When all Optional input instances are missing, partition should not be...
Pallavi Rao [Tue, 21 Jun 2016 04:57:56 +0000 (10:27 +0530)] 
FALCON-2038 When all Optional input instances are missing, partition should not be suffixed.

Updated code and UT.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @PraveenAdlakha, @sandeepSamudrala, @peeyushb

Closes #191 from pallavi-rao/2038

(cherry picked from commit 3abefccf5d311e885051cf7abc6592c19af1b67d)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2036 Update twiki on entity list operation with up-to-date REST API path
yzheng-hortonworks [Tue, 21 Jun 2016 04:54:04 +0000 (10:24 +0530)] 
FALCON-2036 Update twiki on entity list operation with up-to-date REST API path

1. Mark entity type as optional.
2. Reorganize the order of examples.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: @pallavi-rao,@bvellanki

Closes #190 from yzheng-hortonworks/FALCON-2036

(cherry picked from commit dca46a916fb5dba84c324d2a456dd1248a6176b5)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2034 Make numThreads and timeOut configurable In ConfigurationStore init
sandeep [Tue, 21 Jun 2016 04:51:14 +0000 (10:21 +0530)] 
FALCON-2034 Make numThreads and timeOut configurable In ConfigurationStore init

Author: sandeep <sandysmdl@gmail.com>

Reviewers: @pallavi-rao, @peeyushb

Closes #192 from sandeepSamudrala/FALCON-2034 and squashes the following commits:

78b98d5 [sandeep] FALCON-2034. Make numThreads and timeOut configurable In ConfigurationStore init
9d00722 [sandeep] FALCON-2034. Make numThreads and timeOut configurable In ConfigurationStore init

(cherry picked from commit d3ebf0b3d826c32e068c0c786d51dd4d39a7f08f)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2027 Enhance documentation on data replication from HDP to Azure
yzheng-hortonworks [Fri, 17 Jun 2016 18:05:37 +0000 (11:05 -0700)] 
FALCON-2027 Enhance documentation on data replication from HDP to Azure

Also fixed a typo in FalconDocumentation.twiki.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org?"

Closes #187 from yzheng-hortonworks/FALCON-2027

(cherry picked from commit 037e6821b6eb7dbad1c0b1a3508aa6715e77e454)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2031 Hcat Retention test cases are failing NoClassDefFoundError
Peeyush [Thu, 16 Jun 2016 07:03:59 +0000 (12:33 +0530)] 
FALCON-2031 Hcat Retention test cases are failing NoClassDefFoundError

Please review the following pull request, in which we have added the "oozie.action.sharelib.for.java" = "hcatalog,hive" in eviction-action.xml to avoid the failure of HCat retention test cases with "NoClassDefFoundError" when using latest Hive.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com>, Praveen Adlakha <adlakha.praveen@gmail.com>, sandeep <sandysmdl@gmail.com>, Balu <balu@apache.org>

Closes #185 from peeyushb/FALCON-2031

(cherry picked from commit e7cdda742f90744519f522fcf499f79debfeb945)
Signed-off-by: peeyush b <pbishnoi@hortonworks.com>
6 years agoFALCON-2032 Update the extension documentation to add ExtensionService before Configu...
Sowmya Ramesh [Thu, 16 Jun 2016 00:25:51 +0000 (17:25 -0700)] 
FALCON-2032 Update the extension documentation to add ExtensionService before ConfigurationStore in startup properties

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Balu Vellanki <balu@apache.org>

Closes #186 from sowmyaramesh/FALCON-2032

(cherry picked from commit df10d5ac333813edc9a6025b8f3098bd8d3e79db)
Signed-off-by: Sowmya Ramesh <sramesh@hortonworks.com>
6 years agoFALCON-2025 Periodic revalidation of kerberos credentials should be done on loginUser
bvellanki [Tue, 14 Jun 2016 15:57:51 +0000 (08:57 -0700)] 
FALCON-2025 Periodic revalidation of kerberos credentials should be done on loginUser

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "sandeepSamudrala <sandysmdl@gmail.com>, Praveen Adlakha <adlakha.praveen@gmail.com>, Sowmya Ramesh <sowmya_kr@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #183 from bvellanki/FALCON-2025

(cherry picked from commit 00a07d561fdee4aba5be24cfe841c438b17a0e69)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2023 Feed eviction fails when feed locations stats and meta …
Venkatesan Ramachandran [Mon, 13 Jun 2016 22:19:05 +0000 (15:19 -0700)] 
FALCON-2023 Feed eviction fails when feed locations stats and meta …

…does not have time pattern

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: " sandeepSamudrala <sandysmdl@gmail.com>, Ying Zheng <yzheng@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #182 from vramachan/FALCON-2023

6 years agoFALCON-2009 Falcon Aspect weaving issue
Praveen Adlakha [Wed, 8 Jun 2016 18:18:35 +0000 (11:18 -0700)] 
FALCON-2009 Falcon Aspect weaving issue

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Balu <balu@apache.org>"

Closes #178 from PraveenAdlakha/2009

(cherry picked from commit 52425d19c98c160cc1b6fff4065dc10fb9b20a6a)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2008 Add documentation for Graphite Notification Plugin
Praveen Adlakha [Fri, 10 Jun 2016 07:08:21 +0000 (12:38 +0530)] 
FALCON-2008 Add documentation for Graphite Notification Plugin

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: @pallavi-rao

Closes #181 from PraveenAdlakha/2008

(cherry picked from commit 65c804cbac7884f28d60d18116625938a8811456)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2020 Changes in Graphite Notification Plugin
Praveen Adlakha [Fri, 10 Jun 2016 07:05:39 +0000 (12:35 +0530)] 
FALCON-2020 Changes in Graphite Notification Plugin

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: @bvellanki, @pallavi-rao

Closes #180 from PraveenAdlakha/2020 and squashes the following commits:

ecfae9a [Praveen Adlakha] comments addressed
5f3b962 [Praveen Adlakha] FALCON-2020 Changes in Graphite Notification Plugin

(cherry picked from commit 54430ed279909c3e7d488e4ef98d05f56c4ecf45)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2016 maven assembly:single fails on MacOS
Pallavi Rao [Thu, 9 Jun 2016 03:23:07 +0000 (08:53 +0530)] 
FALCON-2016 maven assembly:single fails on MacOS

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @sandeepSamudrala, @bvellanki

Closes #174 from pallavi-rao/2016 and squashes the following commits:

35e1506 [Pallavi Rao] Merge remote-tracking branch 'upstream/master' into 2016
d76f24b [Pallavi Rao] FALCON-2016 maven assembly:single fails on MacOS

(cherry picked from commit b2f085c7cb21b6c5208eaa3aaa8fac331e2cd05a)
Signed-off-by: Pallavi Rao <pallavi.rao@inmobi.com>
6 years agoFALCON-2018 WorkflowJobNotification sends incorrect message for kille…
Praveen Adlakha [Wed, 8 Jun 2016 18:10:15 +0000 (11:10 -0700)] 
FALCON-2018 WorkflowJobNotification sends incorrect message for kille…

…d instances

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #179 from PraveenAdlakha/2018

(cherry picked from commit a4e18a4d428569c37e65e281c3c36bf3ed737eed)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2017 Fix HiveDR extension issues
Sowmya Ramesh [Tue, 7 Jun 2016 21:06:52 +0000 (14:06 -0700)] 
FALCON-2017 Fix HiveDR extension issues

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #175 from sowmyaramesh/FALCON-2017

(cherry picked from commit ee644dd2a6639085b469f8fae96580671f4d0fe9)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1996 Upgrade falcon POM for 0.10 release
bvellanki [Mon, 6 Jun 2016 23:38:35 +0000 (16:38 -0700)] 
FALCON-1996 Upgrade falcon POM for 0.10 release

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #172 from bvellanki/0.10

6 years agoFALCON-1982 Configuration steps for HBase standalone mode
Venkatesan Ramachandran [Mon, 6 Jun 2016 22:21:15 +0000 (15:21 -0700)] 
FALCON-1982 Configuration steps for HBase standalone mode

Step to run HBase in standalone mode to support titan backend.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #164 from vramachan/FALCON-1982.HBaseStandAlone

(cherry picked from commit ea2122b6c699bdb6429e6dda6d91f3d6bdd697d7)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2010 Fix UT errors due to ActiveMQ upgrade
yzheng-hortonworks [Mon, 6 Jun 2016 18:58:56 +0000 (11:58 -0700)] 
FALCON-2010 Fix UT errors due to ActiveMQ upgrade

Fixed two test failures due to the upgrade:

1. JMSMessageConsumerTest
After upgrade, some messages are consumed faster and so the initial expectation on no message being dequeued right after the message sending doesn't hold. Also confirmed with bvellanki that this kind of tests are flaky and have caused intermittent test errors before. Therefore, removed getTotalDequeueCount assertion. In addition, the expected message consumptions are tested in existing code "Mockito.verify(jobEndService,...".

2. FalconPostProcessingTest
After upgrade, Falcon will run into an error if it tries to delete messages when ActiveMQ service is stopped. Therefore, need to call start before deleting the message during teardown.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>, Pallavi Rao <pallavi@apache.org>"

Closes #169 from yzheng-hortonworks/FALCON-2010

(cherry picked from commit fb4d0b4955429e6cef413ab073814fc800361bd4)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1965 Update ActiveMQ version to 5.13.3 to avoid Falcon start error after rolli...
yzheng-hortonworks [Mon, 6 Jun 2016 18:56:59 +0000 (11:56 -0700)] 
FALCON-1965 Update ActiveMQ version to 5.13.3 to avoid Falcon start error after rolling upgrade

Upgraded ActiveMQ version to the latest 5.13.3. Tested Falcon server upgrade with ActiveMQ history works with 0.6, 0.7 and trunk.

Split the fix for two test failures due to upgrade to https://github.com/apache/falcon/pull/169, per vrangan 's suggestion.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>, Pallavi Rao <pallavi@apache.org>"

Closes #168 from yzheng-hortonworks/FALCON-1965

(cherry picked from commit f8707fdc6f512fdfc386b010e4dee0812f45cbd7)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1984 Provide proper hint and documentation if required titan storage backend...
yzheng-hortonworks [Fri, 3 Jun 2016 14:53:16 +0000 (07:53 -0700)] 
FALCON-1984 Provide proper hint and documentation if required titan storage backend is not configured in startup.properties

Added validation of required configuration properties for graph db storage backend.
Added error hints on the missing property in Falcon server log.
Reorganized the order of the configuration properties in startup.properties so relevant properties for each backend are grouped in consecutive lines.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #166 from yzheng-hortonworks/FALCON-1984

(cherry picked from commit e84fca7ff2ce235a7d804b3c4806091a2a2c3037)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1859 Feed export instances are not added to Graph DB
Venkatesan Ramachandran [Fri, 3 Jun 2016 14:50:13 +0000 (07:50 -0700)] 
FALCON-1859 Feed export instances are not added to Graph DB

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #165 from vramachan/FALCON-1859.Export.GraphDB

(cherry picked from commit 16d2b39b24f30b3562874585fcdc58ecf4460ad0)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-1981 Remove runtime jar dependencies from pom files
Venkatesan Ramachandran [Fri, 3 Jun 2016 00:07:51 +0000 (17:07 -0700)] 
FALCON-1981 Remove runtime jar dependencies from pom files

+ Remove jar dependency from pom files that are not required during runtime, but provided by Hadoop
+ End to End test to make sure few flows work

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #161 from vramachan/FALCON-1982.PomCleanUp

(cherry picked from commit 85ed40eba15039c9ed15ab9785f74a991bd50f41)
Signed-off-by: bvellanki <bvellanki@hortonworks.com>
6 years agoFALCON-2007 Hive DR Replication failing with - Can not create a Path from a null...
Peeyush [Thu, 2 Jun 2016 14:41:39 +0000 (20:11 +0530)] 
FALCON-2007 Hive DR Replication failing with - Can not create a Path from a null string

Following is the pull request that will check the existence of the file oozie.action.conf.xml. If file is not there, then it will not get into the code block that adds it to the configuration object, thus to avoid the exception.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Venkat <n.r.v@live.com>

Closes #167 from peeyushb/FALCON-2007

(cherry picked from commit 629bcfbdbd6833cf590a6f622ccde7813e5556dd)
Signed-off-by: peeyush b <pbishnoi@hortonworks.com>
6 years agoFALCON-1893 : Add documentation and examples for spark workflow engine
peeyush b [Mon, 30 May 2016 09:14:42 +0000 (14:44 +0530)] 
FALCON-1893 : Add documentation and examples for spark workflow engine

Please review the pull request which contains documentation with examples how to use the Spark application with Falcon.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Pavan Kumar Kolamuri <pavan.kolamuri@gmail.com>, Venkat Ranganathan <vranganathan@hortonworks.com>"

Closes #150 from peeyushb/FALCON-1893

(cherry picked from commit ff2f9e27fb96e01a1c03598ba936eff77fd3b630)
Signed-off-by: pavankumar526 <pavan.kolamuri@gmail.com>
6 years agoFALCON-1980 Change input and output argument order for Spark process workflow
Peeyush [Mon, 30 May 2016 05:16:02 +0000 (10:46 +0530)] 
FALCON-1980 Change input and output argument order for Spark process workflow

Details to perform this change has been mentioned in the FALCON-1980.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Pavan <pavan.kolamuri@gmail.com>

Closes #160 from peeyushb/FALCON-1980 and squashes the following commits:

23d4e24 [peeyush b] Added comments for input and output argument
3d86f58 [peeyush b] FALCON-1980 : Change input and output argument order for Spark process workflow

(cherry picked from commit 400ef329d1816b408e759990b880c9dd1970dae2)
Signed-off-by: peeyush b <pbishnoi@hortonworks.com>
6 years agoFALCON-1983 Upgrade jackson core and databind versions to fix dependency incompatibil...
yzheng-hortonworks [Fri, 27 May 2016 21:08:33 +0000 (14:08 -0700)] 
FALCON-1983 Upgrade jackson core and databind versions to fix dependency incompatibility with higher-version Hive

Tested it works and generates consistent jackson jars with Hive 1.2.1x.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>,  Venkat Ranganathan <venkat@hortonworks.com>"

Closes #163 from yzheng-hortonworks/FALCON-1983

(cherry picked from commit 2923c04c1d6434cae27f8a4e6b35934d11f1f4ab)
Signed-off-by: Sowmya Ramesh <sramesh@hortonworks.com>
6 years agoFALCON-1985 Disable FeedExportIT and FeedImportIT temporarily
Venkatesan Ramachandran [Thu, 26 May 2016 06:01:26 +0000 (23:01 -0700)] 
FALCON-1985 Disable FeedExportIT and FeedImportIT temporarily

These tests started to fail since may due to some infrastructure changes on the jenkins servers. debugging that will require access to those servers. until then disabling these tests to unblock 0.10 release branch creation.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #162 from vramachan/FALCON-1985.DisableTests

6 years agoFALCON-1858 Support HBase as a storage backend for Falcon Titan graphDB
Venkat Ranganathan [Wed, 25 May 2016 17:15:50 +0000 (22:45 +0530)] 
FALCON-1858 Support HBase as a storage backend for Falcon Titan graphDB

This patch provides the following features
1.   Ability to build Falcon with storage backend plugins without the artifacts (hbase and bdb).
2.   Ability to package a standalone HBase if desired
3.  Backport of Titan 1.0 Hbase 1.0 storage backend into Titan 0.5.4 (shaded classes - mostly similar to Apache Atlas usage of the same) (most of the addons/titan codebase)
4.  Default build profile is with bdb backend with bdb artifacts also included in the build.

Tested the HBase backend with 1.1.5 and 1.2.1 Hbase environments.   Ran test-patch successfully with enabling/disabling of bdb db artifacts and also with all potential addons enabled

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: Peeyush <peeyushb@apache.org>, Balu <balu@apache.org>

Closes #143 from vrangan/FALCON-1858 and squashes the following commits:

693eb05 [Venkat Ranganathan] Disabled hbase also by default and added a comment in startup properties
0b2d8e0 [Venkat Ranganathan] Merged with master with latest changes
7962570 [Venkat Ranganathan] FALCON-1858: Support HBase as a storage backend for Falcon Titan graphDB
a434564 [Venkat Ranganathan] FALCON-1976 :  Remove hadoop-2 profile

6 years agoFALCON-1935 Falcon fails to start with default startup.properties
Praveen Adlakha [Wed, 25 May 2016 13:13:23 +0000 (18:43 +0530)] 
FALCON-1935 Falcon fails to start with default startup.properties

…nJPAService fails)

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Pavan Kumar Kolamuri <pavan.kolamuri@gmail.com"

Closes #128 from PraveenAdlakha/1935

6 years agoFALCON-1979 Update HttpClient versions to close security vulnerabilities
bvellanki [Tue, 24 May 2016 23:49:05 +0000 (16:49 -0700)] 
FALCON-1979 Update HttpClient versions to close security vulnerabilities

Author: bvellanki <bvellanki@hortonworks.com>

Closes #159 from bvellanki/FALCON-1979

6 years agoFALCON-1978 Fix flaky unite test MetadataMappingServicesTests
Venkatesan Ramachandran [Tue, 24 May 2016 21:59:09 +0000 (14:59 -0700)] 
FALCON-1978 Fix flaky unite test MetadataMappingServicesTests

1. Ordered the tests so that parallel execution will not cause assert failures
2. There was an issue found in EntityGraphRelationshipBuilder that was causing MetadataMappingServicesTests.testOnClusterEntityChange(), but it seems that has been fixed in the trunk.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #158 from vramachan/FALCON-1978.FixUnitTest

6 years agoFALCON-1976 and FALCON-1977: Remove hadoop-2 profile and move unit t…
Venkat Ranganathan [Tue, 24 May 2016 02:36:16 +0000 (19:36 -0700)] 
FALCON-1976 and FALCON-1977:  Remove hadoop-2 profile and move unit t…

Removed hadoop-2 profile and moved falcon-unit to addons

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Ying Zheng <yzheng@hortonworks.com>, Peeyush <peeyushb@apache.org>"

Closes #156 from vrangan/FALCON-1976

6 years agoFALCON-1969 Provide server-side error details on CLI, if any
yzheng-hortonworks [Mon, 23 May 2016 19:22:00 +0000 (12:22 -0700)] 
FALCON-1969 Provide server-side error details on CLI, if any

Here is a sample CLI output for comparison.

Before:
_[ambari-qasandbox falcon-0.10-SNAPSHOT]$ bin/falcon extension -submit -extensionName hdfs-mirroring -file /tmp/falcon-xml/hdfs-mirror-para.txt
Hadoop is installed, adding hadoop classpath to falcon classpath
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/falcon-0.10-SNAPSHOT/client/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/falcon-0.10-SNAPSHOT/client/lib/falcon-cli-0.10-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
ERROR: Internal Server Error;_

After:
_[ambari-qasandbox falcon-0.10-SNAPSHOT]$ bin/falcon extension -submit -extensionName hdfs-mirroring -file /tmp/falcon-xml/hdfs-mirror-para.txt
Hadoop is installed, adding hadoop classpath to falcon classpath
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/falcon-0.10-SNAPSHOT/client/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/falcon-0.10-SNAPSHOT/client/lib/falcon-cli-0.10-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.4.0.0-169/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
ERROR: Internal Server Error;**Missing extension property: jobName**_

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #148 from yzheng-hortonworks/FALCON-1969 and squashes the following commits:

2956e89 [yzheng-hortonworks] show error details on CLI
7a76b4a [yzheng-hortonworks] FALCON-1969 Provide server-side error details on CLI, if any

6 years agoFALCON-1963 Falcon CLI should provide detailed hints if the user's command is invalid
yzheng-hortonworks [Mon, 23 May 2016 19:19:31 +0000 (12:19 -0700)] 
FALCON-1963 Falcon CLI should provide detailed hints if the user's command is invalid

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #147 from yzheng-hortonworks/FALCON-1963

6 years agoFALCON-1937 Cluster update documentation
bvellanki [Fri, 20 May 2016 22:14:29 +0000 (15:14 -0700)] 
FALCON-1937 Cluster update documentation

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>"

Closes #153 from bvellanki/FALCON-1937

6 years agoFALCON-1962 Extesnion related bugs
Sowmya Ramesh [Fri, 20 May 2016 20:27:26 +0000 (13:27 -0700)] 
FALCON-1962 Extesnion related bugs

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Venkat <n.r.v@live.com>"

Closes #155 from sowmyaramesh/master

6 years agoFALCON-1972 Handling cases when Extension service or extension.store.uri is not set...
Sowmya Ramesh [Fri, 20 May 2016 17:06:50 +0000 (10:06 -0700)] 
FALCON-1972 Handling cases when Extension service or extension.store.uri is not set in startup properties"

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #152 from sowmyaramesh/FALCON-1972

6 years agoFALCON-1974 Cluster update : Allow superuser to update bundle/coord of dependent...
bvellanki [Fri, 20 May 2016 16:37:51 +0000 (09:37 -0700)] 
FALCON-1974 Cluster update : Allow superuser to update bundle/coord of dependent entities

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng  <yzheng@hortonworks.com>"

Closes #151 from bvellanki/FALCON-1974

6 years agoFALCON-1975 Getting NoSuchMethodError when calling isNoneEmpty
yzheng-hortonworks [Fri, 20 May 2016 15:18:50 +0000 (20:48 +0530)] 
FALCON-1975 Getting NoSuchMethodError when calling isNoneEmpty

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: Balu <bvellanki@hortonworks.com>, Venkat <n.r.v@live.com>, Praveen <adlakha.praveen@gmail.com>, Peeyush <peeyushb@apache.org>

Closes #154 from yzheng-hortonworks/FALCON-1975

6 years agoFALCON-1973 Falcon build failure due to checkstyle issue
Peeyush [Thu, 19 May 2016 12:33:23 +0000 (18:03 +0530)] 
FALCON-1973 Falcon build failure due to checkstyle issue

Please review the following pull request in which Falcon build failure is happening with checkstyle issue from the FALCON-1964 patch.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Pavan <pavan.kolamuri@gmail.com>

Closes #149 from peeyushb/FALCON-1973

6 years agoFALCON-1853 : Add spark process workflow builder
peeyush b [Thu, 19 May 2016 05:19:13 +0000 (10:49 +0530)] 
FALCON-1853 : Add spark process workflow builder

Following is the pull request that will perform the integration to execute the Spark application through Falcon. Please review.

XSD changes for integration are already committed through pull request #89 .

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: "Pavan Kumar Kolamuri <pavan.kolamuri@gmail.com> , Venkat Ranganathan <vranganathan@hortonworks.com>"

Closes #118 from peeyushb/FALCON-1853

6 years agoFALCON-1961 Should return error if an extension job doesn't exist for delete/suspend...
Ying Zheng [Wed, 18 May 2016 22:45:41 +0000 (15:45 -0700)] 
FALCON-1961 Should return error if an extension job doesn't exist for delete/suspend/resume/schedule operations

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Sowmya Ramesh <sowmya_kr@apache.org>, Balu Vellanki <balu@apache.org>"

Closes #144 from yzheng-hortonworks/FALCON-1961

6 years agoFALCON-1964 Should delete temporary JKS file after IT tests for credential provider...
yzheng-hortonworks [Wed, 18 May 2016 20:37:56 +0000 (13:37 -0700)] 
FALCON-1964 Should delete temporary JKS file after IT tests for credential provider alias

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Sowmya Ramesh <sramesh@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #146 from yzheng-hortonworks/FALCON-1964

6 years agoFALCON-1943 Extension API/CLI fails when authorization is enabled
Sowmya Ramesh [Tue, 17 May 2016 21:40:32 +0000 (14:40 -0700)] 
FALCON-1943 Extension API/CLI fails when authorization is enabled

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Ying Zheng <yzheng@hortonworks.com>"

Closes #145 from sowmyaramesh/FALCON-1943

6 years agoFALCON-1956 Graphite Plugin for monitoring
Praveen Adlakha [Tue, 17 May 2016 11:45:04 +0000 (17:15 +0530)] 
FALCON-1956 Graphite Plugin for monitoring

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Pavan Kumar Kolamuri <pavan.kolamuri@gmail.com>, Srikanth Sundarrajan <sriksun@hotmail.com>"

Closes #141 from PraveenAdlakha/codahale

6 years agoFALCON-1957 Documentation on credential provider alias for passwords in startup prope...
yzheng-hortonworks [Mon, 16 May 2016 22:05:45 +0000 (15:05 -0700)] 
FALCON-1957 Documentation on credential provider alias for passwords in startup properties

Also restructured the overview of Falcon security features.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #142 from yzheng-hortonworks/FALCON-1957

6 years agoFALCON-1954 Document enabling Oozie JMS for Falcon
Venkatesan Ramachandran [Fri, 13 May 2016 22:47:13 +0000 (15:47 -0700)] 
FALCON-1954 Document enabling Oozie JMS for Falcon

Steps to enable Oozie JMS notification for Falcon.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #137 from vramachan/FALCON-1948.OozieForFalcon

6 years agoFALCON-1934 Document Safemode in Falcon Server
bvellanki [Fri, 13 May 2016 22:43:59 +0000 (15:43 -0700)] 
FALCON-1934 Document Safemode in Falcon Server

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Venkatesan Ramachandran <vramachandran@hortonworks.com>"

Closes #140 from bvellanki/FALCON-1934

6 years agoFALCON-1919 Provide user the option to store sensitive information with Hadoop creden...
yzheng-hortonworks [Fri, 13 May 2016 22:22:07 +0000 (15:22 -0700)] 
FALCON-1919 Provide user the option to store sensitive information with Hadoop credential provider

Tested alias properties being resolved correctly with Hadoop credential provider.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #138 from yzheng-hortonworks/FALCON-1919

6 years agoFALCON-1908 Document HDFS snapshot based mirroring extension
bvellanki [Fri, 13 May 2016 16:09:41 +0000 (09:09 -0700)] 
FALCON-1908 Document HDFS snapshot based mirroring extension

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>"

Closes #139 from bvellanki/FALCON-1908

6 years agoFALCON-1905 Extension Job Management: IT tests for REST APIs and CLIs
yzheng-hortonworks [Thu, 12 May 2016 18:34:37 +0000 (11:34 -0700)] 
FALCON-1905 Extension Job Management: IT tests for REST APIs and CLIs

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: " Sowmya<sramesh@hortonworks.com>,Balu Vellanki <balu@apache.org>"

Closes #129 from yzheng-hortonworks/FALCON-1905

6 years agoFALCON-1926 Filter out effectively non-falcon related JMS messages …
Venkatesan Ramachandran [Thu, 12 May 2016 01:09:16 +0000 (18:09 -0700)] 
FALCON-1926 Filter out effectively non-falcon related JMS messages …

* Falcon to retrieve Oozie JMS notifications that belong to Falcon generated Workflows .. ignoring notifications generated by other WF directly submitted to Oozie.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Ying Zheng <yzheng@hortonworks.com>"

Closes #119 from vramachan/FALCON-1923.OpsCoord

6 years agoFALCON-1953 Build fails when profiles hivedr and test-patch are used together
bvellanki [Thu, 12 May 2016 00:04:25 +0000 (17:04 -0700)] 
FALCON-1953 Build fails when profiles hivedr and test-patch are used together

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Venkatesan <me.venkatr@gmail.com>"

Closes #136 from bvellanki/FALCON-1953

6 years agoFALCON-1946 Compile time error in build when using JDK1.8
bvellanki [Wed, 11 May 2016 15:55:38 +0000 (08:55 -0700)] 
FALCON-1946 Compile time error in build when using JDK1.8

Author: bvellanki <bvellanki@hortonworks.com>

Closes #135 from bvellanki/FALCON-1946

6 years agoFALCON-1904 Extension Job Management: documentation for REST APIs and CLIs
yzheng-hortonworks [Wed, 11 May 2016 04:36:32 +0000 (21:36 -0700)] 
FALCON-1904 Extension Job Management: documentation for REST APIs and CLIs

Added documentation for all REST APIs and CLIs for extension job management.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Sowmya Ramesh <sramesh@hortonworks.com>"

Closes #131 from yzheng-hortonworks/FALCON-1904

6 years agoFALCON-1942 Allow Falcon server and client classpath to be customizable
Venkat Ranganathan [Tue, 10 May 2016 21:51:05 +0000 (14:51 -0700)] 
FALCON-1942 Allow Falcon server and client classpath to be customizable

Allow server and client classpath to be customizable

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #134 from vrangan/FALCON-1942

6 years agoFALCON-1941 HiveDR fails with NN-HA enabled on both the source and target clusters
Venkat Ranganathan [Tue, 10 May 2016 21:50:02 +0000 (14:50 -0700)] 
FALCON-1941 HiveDR fails with NN-HA enabled on both the source and target clusters

Fix for HiveDR to work NN-HA enabled clusters

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Ying Zheng <yzheng@hortonworks.com>, Balu Vellanki <balu@apache.org>"

Closes #133 from vrangan/FALCON-1941

6 years agoFALCON-1940 Fix misc build issues in Falcon
bvellanki [Tue, 10 May 2016 20:29:38 +0000 (13:29 -0700)] 
FALCON-1940 Fix misc build issues in Falcon

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>"

Closes #132 from bvellanki/FALCON-1940

6 years agoFALCON-1085 Support Cluster entity updates in Falcon Server
bvellanki [Tue, 10 May 2016 20:27:33 +0000 (13:27 -0700)] 
FALCON-1085 Support Cluster entity updates in Falcon Server

Added basic documentation, https://issues.apache.org/jira/browse/FALCON-1937 will contain detailed documentation.

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, yzheng-hortonworks <yzheng@hortonworks.com>, peeyush b <pbishnoi@hortonworks.com>"

Closes #127 from bvellanki/FALCON-1085

6 years agoFALCON-1939 Avoid creating multiple falcon*.tar.gz during falcon build
bvellanki [Mon, 9 May 2016 23:31:29 +0000 (16:31 -0700)] 
FALCON-1939 Avoid creating multiple falcon*.tar.gz during falcon build

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: Venkat Ranganathan <venkat@hortonworks.com>

Closes #130 from bvellanki/FALCON-1939

6 years agoFALCON-1932 Extension CLI should support common options
yzheng-hortonworks [Thu, 5 May 2016 22:55:41 +0000 (15:55 -0700)] 
FALCON-1932 Extension CLI should support common options

Tested CLI command with common options.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #124 from yzheng-hortonworks/FALCON-1932

6 years agoFALCON-1936 Extensions related files are not available in distro packaging
Sowmya Ramesh [Thu, 5 May 2016 18:25:30 +0000 (11:25 -0700)] 
FALCON-1936 Extensions related files are not available in distro packaging

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>"

Closes #125 from sowmyaramesh/FALCON-1936_1

6 years agoFALCON-1106 Documentation for extensions
Sowmya Ramesh [Tue, 3 May 2016 21:45:40 +0000 (14:45 -0700)] 
FALCON-1106 Documentation for extensions

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>", Ying Zheng <yzheng@hortonworks.com>"

Closes #120 from sowmyaramesh/FALCON-1106

6 years agoFALCON-1767 Improve Falcon retention policy documentation
["Sowmya Ramesh [Tue, 3 May 2016 21:21:31 +0000 (14:21 -0700)] 
FALCON-1767 Improve Falcon retention policy documentation

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #121 from sowmyaramesh/FALCON-1767

6 years agoFALCON-1623 Implement safemode in Falcon Server
bvellanki [Tue, 3 May 2016 18:23:43 +0000 (11:23 -0700)] 
FALCON-1623 Implement safemode in Falcon Server

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "yzheng-hortonworks <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #116 from bvellanki/FALCON-1623

6 years agoFALCON-1928 Updating default startup.properties to add FalconJPAService
Pallavi Rao [Thu, 28 Apr 2016 10:42:07 +0000 (16:12 +0530)] 
FALCON-1928 Updating default startup.properties to add FalconJPAService

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @sandeepSamudrala, @peeyushb

Closes #115 from pallavi-rao/1928

6 years ago89
Peeyush Bishnoi [Wed, 27 Apr 2016 16:02:27 +0000 (21:32 +0530)] 
89

Pull request is available that will extend the cluster and process XSD to include the spark execution engine.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Pallavi <pallavi@inmobi.com>, Pavan <pavan.kolamuri@gmail.com>, Venkat <n.r.v@live.com>, Venkatesan <me.venkatr@gmail.com>

Closes #89 from peeyushb/FALCON-1817 and squashes the following commits:

64d8c35 [peeyush b] making name as required parameter
b26af4e [peeyush b] removed line
8056916 [peeyush b] Incorporating review comments

6 years agoFALCON-1909 Catalog instance triage action fails with null pointer exception
bvellanki [Wed, 27 Apr 2016 07:03:34 +0000 (00:03 -0700)] 
FALCON-1909 Catalog instance triage action fails with null pointer exception

…ception.

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: " Praveen Adlakha <adlakha.praveen@gmail.com>, yzheng-hortonworks <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #117 from bvellanki/FALCON-1909