6 years agoFALCON-1767 Improve Falcon retention policy documentation
["Sowmya Ramesh [Tue, 3 May 2016 21:21:31 +0000 (14:21 -0700)] 
FALCON-1767 Improve Falcon retention policy documentation

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #121 from sowmyaramesh/FALCON-1767

6 years agoFALCON-1623 Implement safemode in Falcon Server
bvellanki [Tue, 3 May 2016 18:23:43 +0000 (11:23 -0700)] 
FALCON-1623 Implement safemode in Falcon Server

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "yzheng-hortonworks <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #116 from bvellanki/FALCON-1623

6 years agoFALCON-1928 Updating default startup.properties to add FalconJPAService
Pallavi Rao [Thu, 28 Apr 2016 10:42:07 +0000 (16:12 +0530)] 
FALCON-1928 Updating default startup.properties to add FalconJPAService

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @sandeepSamudrala, @peeyushb

Closes #115 from pallavi-rao/1928

6 years ago89
Peeyush Bishnoi [Wed, 27 Apr 2016 16:02:27 +0000 (21:32 +0530)] 

Pull request is available that will extend the cluster and process XSD to include the spark execution engine.

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Pallavi <pallavi@inmobi.com>, Pavan <pavan.kolamuri@gmail.com>, Venkat <n.r.v@live.com>, Venkatesan <me.venkatr@gmail.com>

Closes #89 from peeyushb/FALCON-1817 and squashes the following commits:

64d8c35 [peeyush b] making name as required parameter
b26af4e [peeyush b] removed line
8056916 [peeyush b] Incorporating review comments

6 years agoFALCON-1909 Catalog instance triage action fails with null pointer exception
bvellanki [Wed, 27 Apr 2016 07:03:34 +0000 (00:03 -0700)] 
FALCON-1909 Catalog instance triage action fails with null pointer exception


Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: " Praveen Adlakha <adlakha.praveen@gmail.com>, yzheng-hortonworks <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #117 from bvellanki/FALCON-1909

6 years agoFALCON-1899 Create examples artifact module in Falcon
Peeyush Bishnoi [Sat, 23 Apr 2016 04:53:44 +0000 (10:23 +0530)] 
FALCON-1899 Create examples artifact module in Falcon

Author: peeyush b <pbishnoi@hortonworks.com>

Reviewers: Balu <bvellanki@hortonworks.com>, Venkatesan <me.venkatr@gmail.com>, Venkat Ranganathan <n.r.v@live.com>

Closes #100 from peeyushb/FALCON-1899 and squashes the following commits:

0824204 [peeyush b] changed examples path in assembly-standalone.xml
cdcb3cd [peeyush b] removed extra line
26bc4f8 [peeyush b] FALCON-1899 : Create examples artifact module in Falcon

6 years agoFALCON-1924 Falcon Coordinator rerun return old workflow id
Praveen Adlakha [Fri, 22 Apr 2016 16:27:00 +0000 (21:57 +0530)] 
FALCON-1924 Falcon Coordinator rerun return old workflow id

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Pallavi<pallavi.rao@inmobi.com>, Sandeep<sandysmdl@gmail.com>, Peeyush<peeyushb@apache.org>

Closes #113 from PraveenAdlakha/1924

6 years agoFALCON-1897 Extension Job Management: CLI support
yzheng-hortonworks [Fri, 22 Apr 2016 16:04:48 +0000 (09:04 -0700)] 
FALCON-1897 Extension Job Management: CLI support

Note: There is a bug to run Falcon CLI after the code refactoring. sowmyaramesh has the patch to fix this and will send a pull request. Tested all the CLI methods with her fix patch.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #103 from yzheng-hortonworks/FALCON-1897

6 years agoFALCON-1925 Add hadoop classpath to falcon client classpath in falcon-config.sh
Venkat Ranganathan [Fri, 22 Apr 2016 14:30:30 +0000 (20:00 +0530)] 
FALCON-1925 Add hadoop classpath to falcon client classpath in falcon-config.sh

This is for recipe submission in a secure cluster

Author: Venkat Ranganathan <venkat@hortonworks.com>
Author: Sowmya Ramesh <sowmya_kr@apache.org>
Author: Praveen Adlakha <adlakha.praveen@gmail.com>
Author: bvellanki <bvellanki@hortonworks.com>
Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: Peeyush<peeyushb@apache.org>, Sandeep<sandysmdl@gmail.com>

Closes #112 from vrangan/master and squashes the following commits:

66907d8 [Venkat Ranganathan] Merge remote-tracking branch 'upstream/master'
4025644 [Venkat Ranganathan] FALCON-1925: Add hadoop classpath to falcon client classpath in falcon-config.sh
608efeb [bvellanki] FALCON-1861 Support HDFS Snapshot based replication in Falcon
9f7fa4b [Sowmya Ramesh] FALCON-1105 REST API support for Server side extensions artifact repository management
ac6dd92 [yzheng-hortonworks] FALCON-1790 CLI support for instance search
4c50d8d [Praveen Adlakha] FALCON-1749 Instance status does not show instances if entity is dele…
157a4f7 [Venkat Ranganathan] Fix for allowing RM principal to be specified in cluster entity
5edcc8f [Venkat Ranganathan] Merge remote-tracking branch 'upstream/master'
bc5a458 [Venkat Ranganathan] Falcon webUI returns 413 (Full head - Request entity too large) error when TLS is enabled in a secure cluster with AD integration

6 years agoFALCON-1916 Allow RM principal to be specified in Cluster entity
Venkat Ranganathan [Thu, 21 Apr 2016 23:50:44 +0000 (16:50 -0700)] 
FALCON-1916 Allow RM principal to be specified in Cluster entity

When we have clusters setup with individual security domains with  cross realm trusts, the RM principal needs to be specified explicitly to disambiguate the realm of the resource manager of the corresponding cluster entities.

This patch allows RM principal to be set and processed from the cluster properties

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Venkatesan Ramachandran <vramachandran@hortonworks.com>"

Closes #111 from vrangan/master

6 years agoFALCON-1861 Support HDFS Snapshot based replication in Falcon
bvellanki [Thu, 21 Apr 2016 23:48:16 +0000 (16:48 -0700)] 
FALCON-1861 Support HDFS Snapshot based replication in Falcon

Documentation will be added in Jira FALCON-1908

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Sowmya <sramesh@hortonworks.com>, sandeepSamudrala <sandysmdl@gmail.com>, Ying Zheng <yzheng@hortonworks.com>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #105 from bvellanki/master

6 years agoFALCON-1105 REST API support for Server side extensions artifact repository management
Sowmya Ramesh [Thu, 21 Apr 2016 22:18:43 +0000 (15:18 -0700)] 
FALCON-1105 REST API support for Server side extensions artifact repository management

REST API support for Server side extensions artifact repository management.
Fixed the warnings in the file.

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu<bvellanki@hortonworks.com>, Ying Zheng<yzheng@hortonworks.com>"

Closes #102 from sowmyaramesh/FALCON-1105

6 years agoFALCON-1790 CLI support for instance search
yzheng-hortonworks [Thu, 21 Apr 2016 22:06:05 +0000 (15:06 -0700)] 
FALCON-1790 CLI support for instance search

Tested CLI support. Also added documentation for both REST api and CLI.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Praveen Adlakha <adlakha.praveen@gmail.com>, Balu Vellanki <balu@apache.org>"

Closes #110 from yzheng-hortonworks/FALCON-1790

6 years agoFALCON-1749 Instance status does not show instances if entity is dele…
Praveen Adlakha [Thu, 21 Apr 2016 10:09:36 +0000 (15:39 +0530)] 
FALCON-1749 Instance status does not show instances if entity is dele…

…ted from one of the colos

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com>

Closes #108 from PraveenAdlakha/1749 and squashes the following commits:

ef50c74 [Praveen Adlakha] minor fix
0cedbb8 [Praveen Adlakha] Merge branch '1749' of github.com:PraveenAdlakha/falcon into 1749
7e25db6 [Praveen Adlakha] FALCON-1749 Instance status does not show instances if entity is deleted from one of the colos

6 years agoFALCON-1335 Backend support of instance search of a group of entities
yzheng-hortonworks [Wed, 20 Apr 2016 18:49:07 +0000 (11:49 -0700)] 
FALCON-1335 Backend support of instance search of a group of entities

Tested REST API: api/instance/search; Added unit test

p.s. Accidentally deleted the previous branch. Previous reviews: https://github.com/apache/falcon/pull/93

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Sowmya <sramesh@hortonworks.com>, Peeyush<peeyushb@apache.org>, Balu <balu@apache.org>"

Closes #106 from yzheng-hortonworks/FALCON-1335

6 years agoFALCON-1917 Fix build failure
Sowmya Ramesh [Wed, 20 Apr 2016 18:42:11 +0000 (11:42 -0700)] 
FALCON-1917 Fix build failure

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Ying Zheng<yzheng@hortonworks.com>, Peeyush Bishnoi<peeyushb@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>, Praveen Adlakha <adlakha.praveen@gmail.com>

Closes #109 from sowmyaramesh/FALCON-1917

6 years agoFALCON-1802 EL Expressions support in Native Scheduler
pavankumar526 [Tue, 19 Apr 2016 06:33:57 +0000 (12:03 +0530)] 
FALCON-1802 EL Expressions support in Native Scheduler

Author: pavankumar526 <pavan.kolamuri@gmail.com>

Reviewers: "sandeepSamudrala <sandysmdl@gmail.com>, Ajay Yadava <ajayyadava@apache.org>"

Closes #57 from pavankumar526/FALCON-1802

6 years agoFALCON-1907 Package new CLI module
Sowmya Ramesh [Mon, 18 Apr 2016 11:42:00 +0000 (17:12 +0530)] 
FALCON-1907 Package new CLI module

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Sandeep<sandysmdl@gmail.com>, Sowmya<sramesh@hortonworks.com>, Peeyush<peeyushb@apache.org>

Closes #104 from sowmyaramesh/FALCON-1907

6 years agoFALCON-1888 Falcon JMS notification details and example
Venkatesan Ramachandran [Mon, 18 Apr 2016 08:45:39 +0000 (14:15 +0530)] 
FALCON-1888 Falcon JMS notification details and example

Updates the operability document to enable Falcon user notifications and example code to read and Map Message from ActiveMQ.

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: Sowmya<sramesh@hortonworks.com>, Peeyush<peeyushb@apache.org>

Closes #101 from vramachan/master and squashes the following commits:

3429177 [Venkatesan Ramachandran] FALCON-1888 : Falcon JMS system and user notification details and sample code
766c130 [Venkatesan Ramachandran] FALCON-1888 : Falcon JMS system and user notification details and sample code
a1fe26e [Venkatesan Ramachandran] git rebase from upstream master to ramachan fork master
3cfc4e0 [Venkatesan Ramachandran] FALCON-1888 : Falcon JMS details and usage
fac28b1 [Venkatesan Ramachandran] Merge
afa5173 [Venkatesan Ramachandran] FALCON-1838:Export instances are not added graph db for lineage tracking - remove call to findVertex(), refactor method name
a886958 [Venkatesan Ramachandran] FALCON-1838:Export instances are not added graph db for lineage tracking

6 years agoFALCON-1789 Extension Job Management: REST API
Ying Zheng [Fri, 15 Apr 2016 18:46:33 +0000 (11:46 -0700)] 
FALCON-1789 Extension Job Management: REST API

Tested REST APIs. Will add IT tests (FALCON-1905) and documentations (FALCON-1904) in separate pull requests.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu <balu@apache.org>, Sowmya <sowmya_kr@apache.org>"

Closes #98 from yzheng-hortonworks/FALCON-1789

6 years agoFALCON-1902 Command line support for server side extension artifacts…
Sowmya Ramesh [Thu, 14 Apr 2016 23:44:57 +0000 (16:44 -0700)] 
FALCON-1902 Command line support for server side extension artifacts…

… repository management

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Balu<bvellanki@hortonworks.com>, Ying Zheng<yzheng@hortonworks.com>, Peeyush Bishnoi<peeyushb@apache.org>

Closes #99 from sowmyaramesh/FALCON-1902 and squashes the following commits:

60dc1ac [Sowmya Ramesh] FALCON-1902: Incorporated review comments
642f0ae [Sowmya Ramesh] FALCON-1902: Incorpoarted review comments
d27e2db [Sowmya Ramesh] FALCON-1902: Command line support for server side extension artifacts repository management

6 years agoFALCON-1901: Fix intermittent IT test failures caused by extension co…
Sowmya Ramesh [Thu, 14 Apr 2016 01:12:11 +0000 (18:12 -0700)] 
FALCON-1901: Fix intermittent IT test failures caused by extension co…

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Balu<bvellanki@hortonworks.com>, Ying Zheng<yzheng@hortonworks.com>

Closes #97 from sowmyaramesh/FALCON-1901

6 years agoFALCON-1895 Refactoring of FalconCLI and FalconClient
Praveen Adlakha [Wed, 13 Apr 2016 15:39:37 +0000 (21:09 +0530)] 
FALCON-1895 Refactoring of FalconCLI and FalconClient

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Peeyush Bishnoi<peeyushb@apache.org>, Balu<bvellanki@hortonworks.com>, Sowmya<sramesh@hortonworks.com>

Closes #94 from PraveenAdlakha/1895 and squashes the following commits:

00a65a2 [Praveen Adlakha] merge conflicts resolved
d069466 [Praveen Adlakha] sync from upstream
4e79efc [Praveen Adlakha] extra space removed
d8bc106 [Praveen Adlakha] FALCON-1895 Refactoring of FalconCLI and FalconClient

6 years agoFALCON-1898 Build fails on assembly:assembly when using maven-assembly-plugin version 2.6
bvellanki [Wed, 13 Apr 2016 02:15:47 +0000 (19:15 -0700)] 
FALCON-1898 Build fails on assembly:assembly when using maven-assembly-plugin version 2.6

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Venkat Ranganathan <venkat@hortonworks.com>, Venkatesan Ramachandran <vramachandran@hortonworks.com>"

Closes #96 from bvellanki/master

6 years agoFALCON-1107 Move trusted extensions processing to server side
Sowmya Ramesh [Tue, 12 Apr 2016 23:05:48 +0000 (16:05 -0700)] 
FALCON-1107 Move trusted extensions processing to server side

Ignore any documentation issues as it will be addressed in https://issues.apache.org/jira/browse/FALCON-1106. Thanks!

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #92 from sowmyaramesh/FALCON-1107

6 years agoFALCON-1892 Remove client side Recipe logic
Sowmya Ramesh [Tue, 12 Apr 2016 23:04:39 +0000 (16:04 -0700)] 
FALCON-1892 Remove client side Recipe logic

Please use this pull request to review. Ignore any documentation issues as it will be addressed in  https://issues.apache.org/jira/browse/FALCON-1106. Thanks!

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: "Balu Vellanki <balu@apache.org>, Venkat Ranganathan <venkat@hortonworks.com>"

Closes #91 from sowmyaramesh/FALCON-1892

6 years agoFALCON-1896 Failure in Falcon build in distro module
Praveen Adlakha [Tue, 12 Apr 2016 16:00:57 +0000 (09:00 -0700)] 
FALCON-1896 Failure in Falcon build in distro module

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: "Balu Vellanki <balu@apache.org>"

Closes #95 from PraveenAdlakha/1896

6 years agoFALCON-1885 SLA monitoring API throws ResultNotFoundException
Praveen Adlakha [Tue, 12 Apr 2016 14:44:19 +0000 (20:14 +0530)] 
FALCON-1885 SLA monitoring API throws ResultNotFoundException

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>

Closes #88 from PraveenAdlakha/1885

6 years agoFALCON-1882 Instance status api not working via prism
Praveen Adlakha [Tue, 12 Apr 2016 14:41:18 +0000 (20:11 +0530)] 
FALCON-1882 Instance status api not working via prism

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava<ajayyadava@apache.org>, Pavan Kumar Kolamuri

Closes #87 from PraveenAdlakha/1882

6 years agoFALCON-1880 To support TDE encryption, add --skipcrccheck to distcp options for HiveDR.
bvellanki [Wed, 6 Apr 2016 22:16:36 +0000 (15:16 -0700)] 
FALCON-1880 To support TDE encryption, add --skipcrccheck to distcp options for HiveDR.

Add --skipcrccheck to distcp options for HiveDR.

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: "Peeyush Bishnoi <peeyushb@apache.org>,  Ajay Yadava <ajaynsit@gmail.com>, Venkatesan Ramachandran <Me.venkatr@gmail.com>"

Closes #82 from bvellanki/master

6 years agoFALCON-1881 Database export does not require fields tag in entity specification
Venkatesan Ramachandran [Wed, 6 Apr 2016 20:39:42 +0000 (13:39 -0700)] 
FALCON-1881 Database export does not require fields tag in entity specification

Database export does not require fields tag in entity specification

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Peeyush Bishnoi <peeyushb@apache.org>, Pallavi Rao <pallavi.rao@inmobi.com>, Balu Vellanki <bvellanki@hortonworks.com>"

Closes #83 from vramachan/FALCON-1881.20160331-1.ExportFieldsList

6 years agoFALCON-1886 Feed sla monitoring does not work across restarts.
Ajay Yadava [Wed, 6 Apr 2016 10:25:18 +0000 (15:55 +0530)] 
FALCON-1886 Feed sla monitoring does not work across restarts.

Removed redundant fields.
Added unit tests.

Author: Ajay Yadava <ajaynsit@gmail.com>

Reviewers: Praveen Adlakha

Closes #86 from ajayyadava/1886

6 years agoFALCON-1845 Retries fix when entity was deleted during instance rerun
pavankumar526 [Wed, 6 Apr 2016 08:41:49 +0000 (14:11 +0530)] 
FALCON-1845 Retries fix when entity was deleted during instance rerun

Author: pavankumar526 <pavan.kolamuri@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Sandeep Samudrala, Pallavi Rao

Closes #62 from pavankumar526/FALCON-1845

6 years agoFALCON-1855 Falcon regression build fails with checkstyle issues
Murali Ramasami [Wed, 6 Apr 2016 08:39:15 +0000 (14:09 +0530)] 
FALCON-1855 Falcon regression build fails with checkstyle issues

Falcon checkstyle is now moved to parent pom because of which regression is failing. Fixing the path

Author: Murali Ramasami <mramasami@Murali-Ramasami-MBP.local>

Reviewers: "Pragya <mittal.pragya23@gmail.com>, Praveen Adlakha <adlakha.praveen@gmail.com>"

Closes #84 from muraliramasami/FALCON-1855

6 years agoFALCON-1883 Falcon regression build fails with minor checkstyle issues
Murali Ramasami [Wed, 6 Apr 2016 08:35:18 +0000 (14:05 +0530)] 
FALCON-1883 Falcon regression build fails with minor checkstyle issues

Fixing the checkstyle issue observed during building the regression trunk

Author: Murali Ramasami <mramasami@Murali-Ramasami-MBP.local>

Reviewers: "Pragya <mittal.pragya23@gmail.com>, Praveen Adlakha <adlakha.praveen@gmail.com>"

Closes #85 from muraliramasami/FALCON-1883

6 years agoFALCON-1866 Bug in JDBCStateStore
Praveen Adlakha [Wed, 6 Apr 2016 08:15:12 +0000 (13:45 +0530)] 
FALCON-1866 Bug in JDBCStateStore

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Pavan Kumar Kolamuri

Closes #80 from PraveenAdlakha/1866

6 years agoFALCON-1856 Fix ProcessInstanceStatusTest
Pragya [Mon, 4 Apr 2016 10:42:45 +0000 (16:12 +0530)] 
FALCON-1856 Fix ProcessInstanceStatusTest

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Murali Ramasami <murali.msse@gmail.com>

Closes #68 from pragya-mittal/ProcessInstanceStatusTest

6 years agoFALCON-1854 : Fixing PrismProcessScheduleTest and NoOutputProcessTest
Murali Ramasami [Mon, 4 Apr 2016 10:21:38 +0000 (15:51 +0530)] 
FALCON-1854 : Fixing PrismProcessScheduleTest and NoOutputProcessTest

Fixing PrismProcessScheduleTest and NoOutputProcessTest

Author: Murali Ramasami <mramasami@hortonworks.com>

Reviewers: Pragya <mittal.pragya23@gmail.com>

Closes #67 from muraliramasami/master

6 years agoFalcon webUI returns 413
Venkat Ranganathan [Thu, 31 Mar 2016 21:50:56 +0000 (14:50 -0700)] 
Falcon webUI returns 413

 (Full head - Request entity too large) error when TLS is enabled in a secure cluster with AD integration)

Fix is to duplicate the SocketConnector change to increase header and request buffer for Jetty based on startup properties

Author: Venkat Ranganathan <venkat@hortonworks.com>

Reviewers: "Ajay Yadava <ajayyadava@apache.org>"

Closes #81 from vrangan/master

6 years agoFALCON-1867 Hardcoded query names in JDBCStateStore
Praveen Adlakha [Tue, 29 Mar 2016 14:19:45 +0000 (19:49 +0530)] 
FALCON-1867 Hardcoded query names in JDBCStateStore

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>

Closes #79 from PraveenAdlakha/1867

6 years agoFALCON-1865 Persist Feed sla data to database
Praveen Adlakha [Mon, 28 Mar 2016 13:38:56 +0000 (19:08 +0530)] 
FALCON-1865 Persist Feed sla data to database

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>

Closes #77 from PraveenAdlakha/feed_alert

6 years agoFALCON-1874 Import and Export fails with HDFS as src/dest
Pallavi Rao [Mon, 28 Mar 2016 06:43:32 +0000 (12:13 +0530)] 
FALCON-1874 Import and Export fails with HDFS as src/dest

1. job-xml was pointing to hive-site.xml even for HDFS, cause workflow to fail.
2. datain element was repeated in export coordinator, causing workflow to fail.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @vramachan, @PraveenAdlakha

Closes #78 from pallavi-rao/FALCON-1874

6 years agoFALCON-1864 Retry event does not get removed from delay queue
Pallavi Rao [Mon, 28 Mar 2016 06:41:35 +0000 (12:11 +0530)] 
FALCON-1864 Retry event does not get removed from delay queue

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @sandeepSamudrala, @pavankumar526, @PraveenAdlakha

Closes #76 from pallavi-rao/1864

6 years agoFALCON-1840 Archive older definition in case of update
Praveen Adlakha [Wed, 23 Mar 2016 15:13:12 +0000 (20:43 +0530)] 
FALCON-1840 Archive older definition in case of update

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Peeyush Bishnoi<peeyushb@apache.org>, Balu<bvellanki@hortonworks.com>, Pallavi<pallavi.rao@inmobi.com>, Sandeep<sandysmdl@gmail.com>

Closes #75 from PraveenAdlakha/1840 and squashes the following commits:

c9c19d8 [Praveen Adlakha] documentation added for archival in delete and update
6c570e3 [Praveen Adlakha] comments addressed
b475fc4 [Praveen Adlakha] IT added
7ba81af [Praveen Adlakha] FALCON-1840 Archive older definition in case of update

6 years agoFALCON-1852 Make optional input to a process truly optional
Pallavi Rao [Tue, 22 Mar 2016 07:15:40 +0000 (12:45 +0530)] 
FALCON-1852 Make optional input to a process truly optional

The following changes have been made:
1. Creation of "empty dir" under the staging path of a cluster (during cluster creation).
2. Modify OozieELExtensions to look for availability flag and use only those instances that have the entire dataset. Use "empty dir" when no instances resolve.
3. Updated UT and IT for additional validation.

Tested manually with and without availabilityFlag supplied.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: @sandeepSamudrala , @sriksun, @pavankumar526, @ajayyadava

Closes #73 from pallavi-rao/1852 and squashes the following commits:

7e087ed [Pallavi Rao] FALCON-1852 Documentation update for optional feeds
fc9c490 [Pallavi Rao] FALCON-1852 Added mockito to the pom
92ad20c [Pallavi Rao] FALCON-1852 Added UTs
533e221 [Pallavi Rao] FALCON-1852 Make optional input to a process truly optional

6 years agoFALCON-1838 Export instances are not added graph db for lineage tracking
Venkatesan Ramachandran [Thu, 17 Mar 2016 16:01:14 +0000 (09:01 -0700)] 
FALCON-1838 Export instances are not added graph db for lineage tracking

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <bvellanki@hortonworks.com>, Ying Zheng <yzheng@hortonworks.com>"

Closes #74 from vramachan/master

6 years agoFALCON-1860 ADFProviderService should be optional as default setting
yzheng-hortonworks [Wed, 16 Mar 2016 21:41:32 +0000 (14:41 -0700)] 
FALCON-1860 ADFProviderService should be optional as default setting

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: "Balu <bvellanki@hortonworks.com>, Venkatesan Ramachandran <Me.venkatr@gmail.com>, Sowmya Ramesh <sowmya_kr@apache.org>"

Closes #70 from yzheng-hortonworks/FALCON-1860

6 years agoFALCON-1334 Improve search performance with Titan graph database indexing
yzheng-hortonworks [Wed, 16 Mar 2016 15:09:14 +0000 (20:39 +0530)] 
FALCON-1334 Improve search performance with Titan graph database indexing

Upgraded Titan to 0.5.4. Added indexing on status and nominal time. May add more later based on our user study on frequent filters.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: Balu <bvellanki@hortonworks.com>, Peeyush Bishnoi <peeyushb@apache.org>

Closes #69 from yzheng-hortonworks/FALCON-1334 and squashes the following commits:

51d5530 [yzheng-hortonworks] keep property falcon.graph.serialize.path (needed in two tests)
7939b81 [yzheng-hortonworks] FALCON-1333 Improve search performance with Titan graph database indexing

6 years agoFALCON-1836 Import from database to HCatalog
Venkatesan Ramachandran [Mon, 14 Mar 2016 21:34:28 +0000 (14:34 -0700)] 
FALCON-1836 Import from database to HCatalog

Author: Venkatesan Ramachandran <vramachandran@hortonworks.com>

Reviewers: "Balu Vellanki <bvellanki@hortonworks.com>, Ajay Yadava <ajayyadava@apache.org>, Peeyush Bishnoi <bpeeyush@yahoo.co.in>"

Closes #61 from vramachan/master

6 years agoFALCON-1847 Execution order not honored when instances are suspended/resumed
Pallavi Rao [Mon, 14 Mar 2016 12:06:08 +0000 (17:36 +0530)] 
FALCON-1847 Execution order not honored when instances are suspended/resumed

Have made the following changes:

1. Order of instances on which we perform action is sorted in desc only for status and params. Else, older instances should be suspended/resumed first.
2. Status is now read from DB and only additional info is obtained from Oozie.
3. Listening to suspend notification too, so further instances can be scheduled.
4. Resume does not immediately resume the instance in order to honor concurrency.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>

Closes #64 from pallavi-rao/1847 and squashes the following commits:

069c223 [Pallavi Rao] FALCON-1847 Fixing the IT
9cd1249 [Pallavi Rao] FALCON-1847 Execution order not honored when instances are suspended/resumed

6 years agoFALCON-1743 Entity summary does not work via prism.
Ajay Yadava [Mon, 14 Mar 2016 10:03:47 +0000 (15:33 +0530)] 
FALCON-1743 Entity summary does not work via prism.

Author: Ajay Yadava <ajaynsit@gmail.com>

Reviewers: Sandeep Samudrala, Pallavi Rao, Deepak Barr

Closes #65 from ajayyadava/1743

6 years agoFALCON-1848 Late rerun has to be a forced reun
Pallavi Rao [Mon, 14 Mar 2016 09:36:22 +0000 (15:06 +0530)] 
FALCON-1848 Late rerun has to be a forced reun

Had accidentally changed it while fixing FALCON-1835

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>

Closes #66 from pallavi-rao/1848

6 years agoFALCON-1846 Fixing EntityDryRunTest
Pragya [Fri, 11 Mar 2016 10:03:35 +0000 (15:33 +0530)] 
FALCON-1846 Fixing EntityDryRunTest

FALCON-1846 Fixing ProcessInstanceStatusTest,ProcessInstanceResumeTest,EntityDryRunTest

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Murali Ramasami <murali.msse@gmail.com>"

Closes #63 from pragya-mittal/testFix-1846

6 years agoFALCON-1839 Test case for APIs for entities scheduled on native scheduler
Pragya [Mon, 7 Mar 2016 06:13:05 +0000 (11:43 +0530)] 
FALCON-1839 Test case for APIs for entities scheduled on native scheduler

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: "Murali Ramasami <murali.msse@gmail.com>, Paul Isaychuk <pisaychuk@apache.org>, Deepak Kumar Barr <deepak.barr@gmail.com>"

Closes #49 from pragya-mittal/native-api and squashes the following commits:

a7baab5 [Pragya] Modified Changes.txt
c800405 [Pragya] Addressed review comments
4fd3cf8 [Pragya] Making waitTillInstanceReachState generic
9009d39 [Pragya] Adding assertion
8b95448 [Pragya] Adding doc for NativeInstanceUtil.waitTillInstancesAreCreated
d244279 [Pragya] Modifying NativeInstanceUtil.waitTillInstancesAreCreated based on review comments
9a882df [Pragya] Modifying totalMinutesToWait  in NativeInstancEUtil to wait for defined time
97a155f [Pragya] Review comments addressed
bbf551a [Pragya] Review commenst addressed
b674558 [Pragya] FALCON-1839 Test case for APIs for entities scheduled on native scheduler
26c462a [Pragya] Merge branch 'master' of https://github.com/apache/falcon
cd0664a [Pragya] Revert "FALCON-1832 Adding tags in SchedulableEntityInstance.compareTo()"
db094ea [Pragya] FALCON-1832 Adding tags in SchedulableEntityInstance.compareTo()
3c302e2 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
f037385 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
4c19ec0 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
3b7fd63 [Pragya] FALCON-1829 Add regression for submit and schedule process on native scheduler (time based)

6 years agoyes
yzheng-hortonworks [Sat, 5 Mar 2016 01:18:38 +0000 (17:18 -0800)] 

Add running, succeeded, failed, killed and suspended instances to titan DB based on JMS notifications on workflow jobs. Instance-entity edge properties (e.g. nominal time, status) are also added for vertex-centric indexing.

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: Balu Vellanki <bvellanki@hortonworks.com>, Venkatesan Ramachandran <Me.venkatr@gmail.com>

Closes #47 from yzheng-hortonworks/FALCON-1111 and squashes the following commits:

f2c9faf [yzheng-hortonworks] review by Balu
ff9556b [yzheng-hortonworks] review from Venky
b76f642 [yzheng-hortonworks] FALCON-1111 Instance update on titan DB based on JMS notifications on workflow jobs

6 years agoFALCON-1825 Process end time not exclusive in case of Native Scheduler
pavankumar526 [Fri, 4 Mar 2016 12:12:22 +0000 (17:42 +0530)] 
FALCON-1825 Process end time not exclusive in case of Native Scheduler

Author: pavankumar526 <pavan.kolamuri@gmail.com>

Reviewers: "Pallavi Rao <pallavi.rao@inmobi.com>"

Closes #59 from pavankumar526/FALCON-1825

6 years agoFALCON-1826 Execution order not honoured when instances are KILLED
Pallavi Rao [Fri, 4 Mar 2016 06:48:26 +0000 (12:18 +0530)] 
FALCON-1826 Execution order not honoured when instances are KILLED

The problem was that the unregister for schedule did not unregister the instance immediately. It would add to instancesToIgnore and would get ignored while scheduling. This causes issues as instancesToIgnore will need to be scanned at more than one place.

The fix is to remove the instance from "awaitingSchedule" list synchronously for unregister.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: "Pavan Kumar Kolamuri <pavan.kolamuri@gmail.com>"

Closes #58 from pallavi-rao/1826 and squashes the following commits:

9ace53d [Pallavi Rao] FALCON-1826 Addressed review comments
dec29c0 [Pallavi Rao] FALCON-1826 Execution order not honoured when instances are KILLED

6 years agoFALCON-1844 Replication to delete files removed in source by default
Pallavi Rao [Thu, 3 Mar 2016 10:30:07 +0000 (16:00 +0530)] 
FALCON-1844 Replication to delete files removed in source by default

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Pavan Kolamuri <pavan.kolamuri@inmobi.com>, Sandeep Samudrala<sandeep.samudrala@inmobi.com>, Peeyush Bishnoi<peeyusb@hortonworks.com>

Closes #60 from pallavi-rao/1844

6 years agoFALCON-1842 Falcon build failed in Jenkins at OozieFeedWorkflowBuilderTest
bvellanki [Tue, 1 Mar 2016 08:09:00 +0000 (13:39 +0530)] 
FALCON-1842 Falcon build failed in Jenkins at OozieFeedWorkflowBuilderTest

…rTest (Balu Vellanki)

Author: bvellanki <bvellanki@hortonworks.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Pallavi Rao

Closes #56 from bvellanki/master

6 years agoFALCON-1841 Grouping test in falcon for running nightly regression(Pragya Mittal)
Pragya [Tue, 1 Mar 2016 06:43:55 +0000 (12:13 +0530)] 
FALCON-1841 Grouping test in falcon for running nightly regression(Pragya Mittal)

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Murali Ramasami <murali.msse@gmail.com>

Closes #51 from pragya-mittal/grouping

6 years agoFALCON-1724 Falcon CLI.twiki in docs folder is not pointed by index page
Praveen Adlakha [Fri, 26 Feb 2016 08:46:28 +0000 (14:16 +0530)] 
FALCON-1724 Falcon CLI.twiki in docs folder is not pointed by index page

Currentlt Faclon CLI.twiki is not pointed by any page we need to remove it to avoid the confusion.

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com>

Closes #50 from PraveenAdlakha/1724 and squashes the following commits:

f4d183e [Praveen Adlakha] pallavi's comment addressed
d586d04 [Praveen Adlakha] FALCON-1724 Falcon CLI.twiki in docs folder is not pointed by index page

6 years agoFALCON-1835 [IMPROVEMENT] Falcon should do coord rerun rather than workflow rerun
Pallavi Rao [Wed, 24 Feb 2016 11:55:23 +0000 (17:25 +0530)] 
FALCON-1835 [IMPROVEMENT] Falcon should do coord rerun rather than workflow rerun

…to ensure concurrency

Patch contains the following changes:
1. Upgrade to Oozie client 4.2 (from 4.1).
2. Remove workflow rerun completely and switch to coord rerun.
3. Introduced parentId in WorkflowExecutionContext in order to store the coord action of a given workflow. This will be used during retry and late data rerun.
4. In cases where parentId cannot be populated (notification from Post processing action will not contain this), rerun method queries Oozie to retrieve the parent id.
5. The rerun method now returns the actual status after rerun, rather than hardcoded “RUNNING”.
6. In 4.2, Oozie does a Yarn job kill to kill any lingering executions of previous run of a workflow. It uses RMClientProxy to do the same. Since in local mode, RPC cannot happen, had to introduce LocalFalconRPCClientFactory to prevent RPC in local mode.

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>, Srikanth Sundarrajan<srikath.sundarrajan@inmobi.com>, Pavan Kolamuri <pavan.kolamuri@inmobi.com>

Closes #48 from pallavi-rao/1835 and squashes the following commits:

0e0b317 [Pallavi Rao] FALCON-1835 Clearing old settings from config retrieved
0c0d2c4 [Pallavi Rao] FALCON-1835 Updated doc.
418ac16 [Pallavi Rao] FALCON-1835 Fixed checkstyle issue
96cd509 [Pallavi Rao] FALCON-1835 Updated rerun method to throw exception when both skip.nodes and failnodes are specified
8b0cb1f [Pallavi Rao] FALCON-1835 Updated rerun method to merge user props and job conf.
43df66d [Pallavi Rao] FALCON-1835 Falcon should do coord rerun rather than workflow rerun to ensure concurrency

6 years agoFALCON-1811 Status API does not honour start option
Praveen Adlakha [Wed, 24 Feb 2016 11:00:46 +0000 (16:30 +0530)] 
FALCON-1811 Status API does not honour start option

$ bin/falcon instance -type process -name process1 -start 2015-10-09T14:00Z -status -numResults 2
Consolidated Status: SUCCEEDED

Instance Cluster SourceCluster Status Start End Details Log
2015-12-31T01:00Z local - SUSPENDED 2016-02-11T09:34Z - - http://blr-test-129.corp.inmobi.com:11000/oozie?job=0000028-151229192519233-oozie-oozi-W
 mr-node null null
 failed-post-processing null null
2016-01-01T01:00Z local - SUSPENDED 2016-02-11T10:18Z - - http://blr-test-129.corp.inmobi.com:11000/oozie?job=0000030-151229192519233-oozie-oozi-W
 mr-node null null
 failed-post-processing null null

Additional Information:
Response: default/STATUS
Request Id: default/124451500qtp-822600088-12 - 2ca2d36d-6e03-4e14-9e97-1448aed90084

Note we will not get the same result when status is waiting as at that time start time is null.

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com>

Closes #40 from PraveenAdlakha/1811 and squashes the following commits:

d9925d0 [Praveen Adlakha] pallavi's comments addressed
7784468 [Praveen Adlakha] checkstyle voilation issue solved
4530233 [Praveen Adlakha] changes done as per pallavi's comment
3c3da61 [Praveen Adlakha] Deepak's comments addressed
de08573 [Praveen Adlakha] FALCON-1811 Status API does not honour start option

6 years agoFALCON-1621 [BUG] Lifecycle of entity gets missed when prism and falcon server commun...
Praveen Adlakha [Fri, 19 Feb 2016 12:39:19 +0000 (18:09 +0530)] 
FALCON-1621 [BUG] Lifecycle of entity gets missed when prism and falcon server communicates

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>, Pallavi Rao <pallavi.rao@inmobi.com>

Closes #46 from PraveenAdlakha/1621

6 years agoUpdate CHANGES.txt with FALCON-1627 subject
peeyush b [Fri, 19 Feb 2016 05:02:56 +0000 (10:32 +0530)] 
Update CHANGES.txt with FALCON-1627 subject

6 years agoFALCON-1627 Provider integration with Azure Data Factory pipelines
yzheng-hortonworks [Fri, 19 Feb 2016 04:35:27 +0000 (10:05 +0530)] 
FALCON-1627 Provider integration with Azure Data Factory pipelines

This integration allows Microsoft Azure Data Factory pipelines to invoke Falcon activities (i.e. replication, hive and pig proessing work), so the user can build a hybrid Hadoop data pipelines leveraging on-premises Hadoop clusters and cloud based Cortana Analytics services like HDInsight Hadoop clusters and Azure Machine Learning.

Incorporated review comments from Balu on apache review board:

Author: yzheng-hortonworks <yzheng@hortonworks.com>

Reviewers: Sandeep Samudrala<sandysmdl@gmail.com>, Peeyush Bishnoi<peeyushb@apache.org>, Balu Vellanki<bvellanki@hortonworks.com>, Venkat Ranganathan<n.r.v@live.com>

Closes #27 from yzheng-hortonworks/FALCON-1627 and squashes the following commits:

9e13745 [yzheng-hortonworks] throw exception if no super user for adf in the startup properties
63eedfd [yzheng-hortonworks] polish log (review from peeyushb)
e5a1d97 [yzheng-hortonworks] review from PraveenAdlakha and peeyushb
2090aaf [yzheng-hortonworks] review from sandeepSamudrala
4c5f2b2 [yzheng-hortonworks] FALCON-1627 Provider integration with Azure Data Factory pipelines

6 years ago FALCON-1584 Falcon allows invalid hadoop queue name for schedulable feed entities...
bvellanki [Thu, 18 Feb 2016 22:16:55 +0000 (14:16 -0800)] 
 FALCON-1584 Falcon allows invalid hadoop queue name for schedulable feed entities. Contributed by Venkatesan Ramachandran via Balu Vellanki.

6 years agoFALCON-1566 Add test for SLA monitoring API
Pragya [Thu, 18 Feb 2016 09:50:02 +0000 (15:20 +0530)] 
FALCON-1566 Add test for SLA monitoring API

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Paul Isaychuk <pisaychuk@apache.org>, Deepak Kumar Barr <deepak.barr@gmail.com>

Closes #44 from pragya-mittal/feed-sla and squashes the following commits:

c6c6f65 [Pragya] Review comments addressed
251d404 [Pragya] Resolved merge conflicts
f5b2888 [Pragya] FALCON-1566 Add test for SLA monitoring API
f037385 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
4c19ec0 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
3b7fd63 [Pragya] FALCON-1829 Add regression for submit and schedule process on native scheduler (time based)

6 years agoFALCON-887 Support for multiple lib paths in falcon process
Sowmya Ramesh [Thu, 18 Feb 2016 08:55:03 +0000 (14:25 +0530)] 
FALCON-887 Support for multiple lib paths in falcon process

Author: Sowmya Ramesh <sramesh@hortonworks.com>

Reviewers: Peeyush Bishnoi <peeyushb@apache.org>, Pavan Kumar<pavankumar@apache.org>

Closes #38 from sowmyaramesh/FALCON-887 and squashes the following commits:

95865df [Sowmya Ramesh] Moved getting lib path to private method
3209a7e [Sowmya Ramesh] Applied feedback
aa8f6a2 [Sowmya Ramesh] FALCON-887 Fixed checkstyle errors
ad4d455 [Sowmya Ramesh] FALCON-887 Support for multiple lib paths in falcon process

6 years agoFALCON-1567 Test case for Lifecycle feature
Pragya [Wed, 17 Feb 2016 05:15:39 +0000 (10:45 +0530)] 
FALCON-1567 Test case for Lifecycle feature

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Paul Isaychuk <pisaychuk@apache.org>

Closes #43 from pragya-mittal/FALCON-1567 and squashes the following commits:

9d46ea4 [Pragya] FALCON-1567 Test case for Lifecycle feature
cc84d5a [Pragya] Merge branch 'master' of https://github.com/apache/falcon
f037385 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
4c19ec0 [Pragya] Merge branch 'master' of https://github.com/apache/falcon
3b7fd63 [Pragya] FALCON-1829 Add regression for submit and schedule process on native scheduler (time based)

6 years agoFALCON-1784 Add regression test for for FALCON-1647. Contributed by Paul Isaychuk
Paul Isaychuk [Wed, 27 Jan 2016 15:42:32 +0000 (17:42 +0200)] 
FALCON-1784 Add regression test for for FALCON-1647. Contributed by Paul Isaychuk

6 years agoFALCON-1831 [TEST] Flaky WorkflowExecutionContextTest.testWorkflowStartEnd
Pallavi Rao [Tue, 16 Feb 2016 04:35:47 +0000 (10:05 +0530)] 
FALCON-1831 [TEST] Flaky WorkflowExecutionContextTest.testWorkflowStartEnd

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>, Pavan Kolamuri <pavan.kolamuri@inmobi.com>

Closes #42 from pallavi-rao/1831

6 years agoFALCON-1496 [TEST] Flaky FalconPostProcessingTest
Pallavi Rao [Tue, 16 Feb 2016 04:33:09 +0000 (10:03 +0530)] 
FALCON-1496 [TEST] Flaky FalconPostProcessingTest

Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Deeepak Barr <deepak.barr@gmail.com>

Closes #41 from pallavi-rao/1496

6 years agoFALCON-1823 [BUG] wrong permissions on folders in distributed mode deb
Praveen Adlakha [Mon, 15 Feb 2016 10:01:16 +0000 (15:31 +0530)] 
FALCON-1823 [BUG] wrong permissions on folders in distributed mode deb

Testing :


If we extract the tar in target and run mvn clean install we get error :

rm: cannot remove ‘falcon-distributed-0.10-SNAPSHOT/hadooplibs/curator-framework-2.6.0.jar’: Permission denied
drw-r--r-- 2 praveen praveen 4.0K Feb  5 16:20 conf

There was no read permissions for files under conf because of which prism was not able to start.


drwxr--r-- 2 dataqa dataqa 4.0K Feb  5 16:20 conf

$ bin/prism-start
Hadoop is installed, adding hadoop classpath to falcon classpath
prism started using hadoop version:  Hadoop
$ bin/falcon-start
Hadoop is installed, adding hadoop classpath to falcon classpath
falcon started using hadoop version:  Hadoop
$ bin/falcon-status
Hadoop is installed, adding hadoop classpath to falcon classpath
falcon process is running with PID: 31752
$ bin/prism-status
Hadoop is installed, adding hadoop classpath to falcon classpath
prism process is running with PID: 3167

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Peeyush Bishnoi <peeyushb@hortonworks.com>, Pallavi Rao <pallavi.rao@inmobi.com>

Closes #36 from PraveenAdlakha/1823

6 years agoFALCON-1819 Improve test class entity cleanup logic
Paul Isaychuk [Fri, 12 Feb 2016 10:01:33 +0000 (12:01 +0200)] 
FALCON-1819 Improve test class entity cleanup logic

Improve entity clean up logic to handle entities belonging to different users.

Author: Paul Isaychuk <pisaychuk@apache.org>

Reviewers: Pragya Mittal <mittal.pragya23@gmail.com>

Closes #30 from pisaychuk/master

6 years agoFALCON-1829 [TEST] Add regression for submit and schedule process on native
Pragya [Thu, 11 Feb 2016 12:18:32 +0000 (17:48 +0530)] 
FALCON-1829 [TEST] Add regression for submit and schedule process on native

…scheduler (time based)

Author: Pragya <mittal.pragya23@gmail.com>

Reviewers: Paul Isaychuk <pisaychuk@hortonworks.com>

Closes #39 from pragya-mittal/master

6 years agoFALCON-1783 Fix ProcessUpdateTest and SearchApiTest to use prism 37/head
Paul Isaychuk [Tue, 9 Feb 2016 13:13:16 +0000 (15:13 +0200)] 
FALCON-1783 Fix ProcessUpdateTest and SearchApiTest to use prism

We have some weak tests which work in embedded mode, but fail in distributed. Issue exists because in case of distributed mode, entity submit is not allowed via server

Author: Paul Isaychuk <pisaychuk@apache.org>

Reviewers: Pragya Mittal <mittal.pragya23@gmail.com>

Closes #35 from pisaychuk/FALCON-1783

6 years agoFALCON-1774 Server side check added to disallow Prism operations
Praveen Adlakha [Mon, 8 Feb 2016 08:52:29 +0000 (14:22 +0530)] 
FALCON-1774 Server side check added to disallow Prism operations

Testing :

$ bin/falcon entity -type feed -submitAndSchedule -file ../cluster.xml
ERROR: Forbidden;submitAndSchedule on server is not supported.Please run your operation on Prism.

$ bin/falcon entity -type feed -update -name abc -file ../cluster.xml
ERROR: Forbidden;update on server is not supported.Please run your operation on Prism.

$ bin/falcon entity -type feed -delete -name abc
ERROR: Forbidden;delete on server is not supported.Please run your operation on Prism.

$ bin/falcon entity -type feed -submit -file ../cluster.xml
ERROR: Forbidden;submit on server is not supported.Please run your operation on Prism.

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com> Pavan Kolamuri <pavan.kolamuri@inmobi.com>

Closes #16 from PraveenAdlakha/FALCON-1774 and squashes the following commits:

4935e2b [Praveen Adlakha] rebased patched
23ba055 [Praveen Adlakha] checkstyle issue fixed
75807f6 [Praveen Adlakha] changes.txt updated

6 years agoFALCON-1795 Kill api not killing waiting/ready instances
sandeep [Fri, 5 Feb 2016 15:36:39 +0000 (21:06 +0530)] 
FALCON-1795 Kill api not killing waiting/ready instances

Author: sandeep <sandysmdl@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Pavan Kumar, Peeyush Bishnoi

Closes #22 from sandeepSamudrala/master

6 years agoFALCON-1804 Non-SLA feed throws NullPointerException
sandeep [Fri, 5 Feb 2016 10:43:52 +0000 (16:13 +0530)] 
FALCON-1804 Non-SLA feed throws NullPointerException

Author: sandeep <sandysmdl@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Pavan Kumar Kolamuri, Praveen Adlakha

Closes #21 from sandeepSamudrala/FALCON-1721

6 years agoFALCON-1818 : Minor doc update for tar package locations after FALCON - 1751
Deepak Kumar Barr [Fri, 5 Feb 2016 09:09:06 +0000 (14:39 +0530)] 
FALCON-1818 : Minor doc update for tar package locations after FALCON - 1751

Author: Deepak Kumar Barr <deepak.barr@gmail.com>

Reviewers: Ajay Yadava <ajayyadava@apache.org>, Balu Vellanki, Pallavi Rao

Closes #24 from deepakbarr/falcon-1818

6 years agoFALCON-1806 : Update documentation for Import and Export. Contributed by Venkatesan...
bvellanki [Thu, 4 Feb 2016 23:02:17 +0000 (15:02 -0800)] 
FALCON-1806 : Update documentation for Import and Export. Contributed by Venkatesan Ramachandran

6 years ago FALCON-1729 Database ingest to support password alias via keystore file ( Contribute...
bvellanki [Thu, 4 Feb 2016 21:16:29 +0000 (13:16 -0800)] 
 FALCON-1729 Database ingest to support password alias via keystore file ( Contributed by Venkatesan Ramachandran)

6 years agoFALCON-1787 Ooozie pig-action.xml requires hive sharelib for HCatalog use. Contribute...
Ajay Yadava [Thu, 4 Feb 2016 08:26:43 +0000 (13:56 +0530)] 
FALCON-1787 Ooozie pig-action.xml requires hive sharelib for HCatalog use. Contributed by Sowmya Ramesh.

6 years agoFALCON-1816 Fix findbugs-exclude.xml path and hadoop version in falcon-regression...
Ajay Yadava [Thu, 4 Feb 2016 08:12:46 +0000 (13:42 +0530)] 
FALCON-1816 Fix findbugs-exclude.xml path and hadoop version in falcon-regression pom. Contributed by Paul Isaychuk.

6 years agoFALCON-1793 feed element action="archive" is submittable via command line tool falcon
Deepak Kumar Barr [Wed, 3 Feb 2016 04:12:55 +0000 (09:42 +0530)] 
FALCON-1793 feed element action="archive" is submittable via command line tool falcon

Changed as per review comments.

$  bin/falcon entity -submit -type feed -file examples/entity/filesystem/out-feed.xml
ERROR: Bad Request;javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 29; columnNumber: 59; cvc-enumeration-valid: Value 'archive' is not facet-valid with respect to enumeration '[delete]'. It must be a value from the enumeration.]

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] Apache Falcon ...................................... SUCCESS [  2.459 s]
[INFO] Apache Falcon UI ................................... SUCCESS [  8.276 s]
[INFO] Build Tools ........................................ SUCCESS [  5.683 s]
[INFO] Apache Falcon Java client .......................... SUCCESS [ 13.753 s]
[INFO] Apache Falcon CLI client ........................... SUCCESS [  4.490 s]
[INFO] Apache Falcon Metrics .............................. SUCCESS [  6.040 s]
[INFO] Apache Falcon Hadoop Dependencies .................. SUCCESS [  3.137 s]
[INFO] Apache Falcon Test Utility ......................... SUCCESS [  4.829 s]
[INFO] Apache Falcon Commons .............................. SUCCESS [02:52 min]
[INFO] Apache Falcon Oozie EL Extension ................... SUCCESS [  7.907 s]
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ....... SUCCESS [ 28.412 s]
[INFO] Apache Falcon Sharelib Hive - Test Cluster ......... SUCCESS [  3.311 s]
[INFO] Apache Falcon Sharelib Pig - Test Cluster .......... SUCCESS [  0.800 s]
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster ..... SUCCESS [  2.140 s]
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ........ SUCCESS [  0.168 s]
[INFO] Apache Falcon Test Tools - Test Cluster ............ SUCCESS [  0.115 s]
[INFO] Apache Falcon Messaging ............................ SUCCESS [ 21.885 s]
[INFO] Apache Falcon LIfecycle Module ..................... SUCCESS [  7.531 s]
[INFO] Apache Falcon Oozie Adaptor ........................ SUCCESS [01:08 min]
[INFO] Apache Falcon Scheduler ............................ SUCCESS [01:25 min]
[INFO] Apache Falcon Acquisition .......................... SUCCESS [  1.062 s]
[INFO] Apache Falcon Distcp Replication ................... SUCCESS [  8.399 s]
[INFO] Apache Falcon Retention ............................ SUCCESS [ 13.447 s]
[INFO] Apache Falcon Archival ............................. SUCCESS [  0.505 s]
[INFO] Apache Falcon Rerun ................................ SUCCESS [ 15.544 s]
[INFO] Apache Falcon Prism ................................ SUCCESS [01:34 min]
[INFO] falcon-unit ........................................ SUCCESS [02:48 min]
[INFO] Apache Falcon Web Application ...................... SUCCESS [07:42 min]
[INFO] Apache Falcon Documentation ........................ SUCCESS [  6.900 s]
[INFO] Apache Falcon Distro ............................... SUCCESS [04:27 min]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24:48 min
[INFO] Finished at: 2016-02-03T01:55:36+05:30
[INFO] Final Memory: 719M/2208M
[INFO] ------------------------------------------------------------------------

Author: Deepak Kumar Barr <deepak.barr@gmail.com>

Reviewers: Pallavi Rao <pallavi.rao@inmobi.com>, Ajay Yadava <ajaynsit@gmail.com>

Closes #23 from deepakbarr/falcon-1793-2

6 years agoFALCON-1792 Upgrade hadoop.version to 2.6.2. Contributed by Venkatesan Ramachandran.
peeyush b [Tue, 2 Feb 2016 15:39:23 +0000 (21:09 +0530)] 
FALCON-1792 Upgrade hadoop.version to 2.6.2. Contributed by Venkatesan Ramachandran.

6 years agoFALCON-1766 Add CLI metrics check for HiveDR, HDFS and feed replication. Contributed...
Paul Isaychuk [Thu, 21 Jan 2016 15:03:55 +0000 (17:03 +0200)] 
FALCON-1766 Add CLI metrics check for HiveDR, HDFS and feed replication. Contributed by Paul Isaychuk

6 years agoFALCON-1721 Move checkstyle under parent pom
sandeep [Tue, 2 Feb 2016 04:41:56 +0000 (10:11 +0530)] 
FALCON-1721 Move checkstyle under parent pom

Author: sandeep <sandysmdl@gmail.com>

Reviewers: praveen <praveen.adlakha@inmobi.com>, Pallavi Rao <pallavi.rao@inmobi.com

Closes #19 from sandeepSamudrala/master

6 years agoFALCON-1796 [HOTFIX] Incorrect parent pom in distro module. Contributed by Ajay Yadava.
Ajay Yadav [Mon, 1 Feb 2016 09:15:13 +0000 (14:45 +0530)] 
FALCON-1796 [HOTFIX] Incorrect parent pom in distro module. Contributed by Ajay Yadava.

6 years agoFALCON-1751 Support assembly:single mojo. Contributed by Ruoyu Wang.
Ajay Yadava [Fri, 29 Jan 2016 20:34:35 +0000 (02:04 +0530)] 
FALCON-1751 Support assembly:single mojo. Contributed by Ruoyu Wang.

6 years agoFALCON-1782 Client returns FalconWebException instead of the expected…
Praveen Adlakha [Fri, 29 Jan 2016 14:41:42 +0000 (20:11 +0530)] 
FALCON-1782 Client returns FalconWebException instead of the expected…

testing :

$ bin/falcon entity -type process -name process1 -properties falcon.scheduler:native -schedule
ERROR: Bad Request;default/Entity process1 is already scheduled on configured workflow engine.

Author: Praveen Adlakha <adlakha.praveen@gmail.com>

Reviewers: Ajay Yadava<ajayyadava@apache.org>, Sandeep Samudrala <sandysmdl@gmail.com>

Closes #18 from PraveenAdlakha/FALCON-1782

6 years agoFALCON-1750 Null Pointer Exception while listening to Workflow Notifi…
Pallavi Rao [Fri, 29 Jan 2016 11:47:05 +0000 (17:17 +0530)] 
FALCON-1750 Null Pointer Exception while listening to Workflow Notifi…


Author: Pallavi Rao <pallavi.rao@inmobi.com>

Reviewers: Sandeep Samudrala <sandeep.samudrala@inmobi.com>, Pavan Kolamuri <pavan.kolamuri@inmobi.com>

Closes #17 from pallavi-rao/master and squashes the following commits:

ae8dacb [Pallavi Rao] FALCON-1750 Addressed review comments
9e95e12 [Pallavi Rao] Merge remote-tracking branch 'upstream/master'
b04519e [Pallavi Rao] FALCON-1750 Null Pointer Exception while listening to Workflow Notifications

6 years agoFALCON-1778 Check licenses and copyright information (By Pallavi Rao)
Pallavi Rao [Fri, 29 Jan 2016 09:45:55 +0000 (15:15 +0530)] 
FALCON-1778 Check licenses and copyright information (By Pallavi Rao)

6 years agoFALCON-763 Support feed listing for CatalogStorage. Contributed by Balu Vellanki
bvellanki [Thu, 28 Jan 2016 19:48:30 +0000 (11:48 -0800)] 
FALCON-763 Support feed listing for CatalogStorage. Contributed by Balu Vellanki

6 years agoFALCON-1777 Add regression for HDFS replication (recipe). Contributed by Paul Isaychuk
Paul Isaychuk [Thu, 28 Jan 2016 15:37:40 +0000 (17:37 +0200)] 
FALCON-1777 Add regression for HDFS replication (recipe). Contributed by Paul Isaychuk

6 years agoRemove temporary folder "localhost" created during tests. Contributed by Praveen...
Ajay Yadava [Wed, 27 Jan 2016 19:50:13 +0000 (01:20 +0530)] 
Remove temporary folder "localhost" created during tests. Contributed by Praveen Adlakha.

6 years agoFALCON-1756 Remove PID files on service stop. Contributed by Deepak Barr.
Ajay Yadava [Wed, 27 Jan 2016 19:46:25 +0000 (01:16 +0530)] 
FALCON-1756 Remove PID files on service stop. Contributed by Deepak Barr.

6 years agoFALCON-1771 Tool to merge pull requests. Contributed by Ajay Yadava.
Ajay Yadava [Wed, 27 Jan 2016 19:40:34 +0000 (01:10 +0530)] 
FALCON-1771 Tool to merge pull requests. Contributed by Ajay Yadava.

6 years agoFALCON-1779 Fix rat-check failure in trunk. Contributed by Ajay Yadava.
Ajay Yadava [Wed, 27 Jan 2016 19:01:04 +0000 (00:31 +0530)] 
FALCON-1779 Fix rat-check failure in trunk. Contributed by Ajay Yadava.