incubator-amaterasu.git
2 weeks agoMerge pull request #37 from ebarten/feature/AMATERASU-51 master
Yaniv Rodenski [Sun, 25 Nov 2018 13:33:43 +0000 (00:33 +1100)] 
Merge pull request #37 from ebarten/feature/AMATERASU-51

feature/amaterasu-51: Add support for configuring the path for action level environment path

2 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path 37/head
Eran Bartenstein [Sun, 25 Nov 2018 10:11:22 +0000 (21:11 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Some code cleanup: remove unused imports

2 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Sun, 25 Nov 2018 10:10:19 +0000 (21:10 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Some code cleanup

2 weeks agofixed config parsing
Yaniv Rodenski [Sun, 25 Nov 2018 04:51:20 +0000 (15:51 +1100)] 
fixed config parsing

2 weeks agoMerge branch 'feature/AMATERASU-51' of github.com:ebarten/incubator-amaterasu into...
Yaniv Rodenski [Sun, 25 Nov 2018 04:21:32 +0000 (15:21 +1100)] 
Merge branch 'feature/AMATERASU-51' of github.com:ebarten/incubator-amaterasu into feature/AMATERASU-51

2 weeks agotests are now fixed
Yaniv Rodenski [Sun, 25 Nov 2018 04:21:21 +0000 (15:21 +1100)] 
tests are now fixed

2 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Sat, 24 Nov 2018 00:06:20 +0000 (11:06 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Add test for JobParserTests to test config is read correctly.

2 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Fri, 23 Nov 2018 23:58:55 +0000 (10:58 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
PArse action by JobParser and add it as a parameter to ActionData so that it could receive the parsed input from JobParser via SequentialAction

3 weeks agonew kotlin based jpb parser now builds
Yaniv Rodenski [Thu, 15 Nov 2018 01:57:10 +0000 (12:57 +1100)] 
new kotlin based jpb parser now builds

3 weeks agofixed license headers
Yaniv Rodenski [Wed, 14 Nov 2018 02:47:25 +0000 (13:47 +1100)] 
fixed license headers

3 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Wed, 14 Nov 2018 01:47:13 +0000 (12:47 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Move JobParser from Scala -> Kotlin & into leader-common

3 weeks agoMerge remote-tracking branch 'remotes/origin/feature/AMATERASU-56' into feature/AMATE...
Eran Bartenstein [Tue, 13 Nov 2018 06:45:03 +0000 (17:45 +1100)] 
Merge remote-tracking branch 'remotes/origin/feature/AMATERASU-56' into feature/AMATERASU-51

3 weeks agocrated kotlin SequentialAction and ErrorAction implementations
Yaniv Rodenski [Tue, 13 Nov 2018 00:35:02 +0000 (11:35 +1100)] 
crated kotlin SequentialAction and ErrorAction implementations

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Sat, 10 Nov 2018 09:46:50 +0000 (20:46 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Move JobManager from Scala -> Kotlin

4 weeks agoMerge pull request #36 from ebarten/feature/AMATERASU-56
Yaniv Rodenski [Fri, 9 Nov 2018 00:04:13 +0000 (11:04 +1100)] 
Merge pull request #36 from ebarten/feature/AMATERASU-56

Amaterasu-56: Create a Kotlin logger implementation

4 weeks agofixed the conversion between scala and kotlin for ActionData in the tests 36/head
Yaniv Rodenski [Thu, 8 Nov 2018 23:20:35 +0000 (10:20 +1100)] 
fixed the conversion between scala and kotlin for ActionData in the tests

4 weeks agofixed the conversion between scala and kotlin for ActionData
Yaniv Rodenski [Thu, 8 Nov 2018 22:54:04 +0000 (09:54 +1100)] 
fixed the conversion between scala and kotlin for ActionData

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Thu, 8 Nov 2018 07:58:29 +0000 (18:58 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Pull latest from apache master and merge latest changes

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Thu, 8 Nov 2018 01:12:37 +0000 (12:12 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Fix build.gradle Kotlin plugin id version -> 1.3.0

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Thu, 8 Nov 2018 01:01:38 +0000 (12:01 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Move logging into Kotlin and create a bridge between Scala and Kotlin via Java

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Mon, 5 Nov 2018 08:36:33 +0000 (19:36 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Sort out compilation order for 'common'

4 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Mon, 5 Nov 2018 08:33:47 +0000 (19:33 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Convert Logging into Kotlin and create a Java proxy class

5 weeks agoAction Kotlin refactoring fixed
Yaniv Rodenski [Sun, 4 Nov 2018 02:13:26 +0000 (13:13 +1100)] 
Action Kotlin refactoring fixed

5 weeks agoseprate import statment while scala and kotlin mixed togther
guy.peleg [Wed, 31 Oct 2018 03:08:28 +0000 (14:08 +1100)] 
seprate import statment while scala and kotlin mixed togther

5 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Wed, 31 Oct 2018 02:37:34 +0000 (13:37 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Resolve further compilation issues with scala to Kotlin conversion
CODE still does NOT compile.

5 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Tue, 30 Oct 2018 22:29:46 +0000 (09:29 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Complete conversion of Action.scala to Kotlin

5 weeks agofixing Scala to Kotlin stuff
guy.peleg [Tue, 30 Oct 2018 10:01:32 +0000 (21:01 +1100)] 
fixing Scala to Kotlin stuff

6 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Thu, 25 Oct 2018 06:06:31 +0000 (17:06 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Convert Action, ActionData & ActionStatus to Kotlin

7 weeks agoMerge pull request #35 from roadan/AMATERASU-48
Nadav Har Tzvi [Sun, 21 Oct 2018 03:33:06 +0000 (06:33 +0300)] 
Merge pull request #35 from roadan/AMATERASU-48

Adding getActionResources and getActionDependencies

7 weeks agoadded getActionResources and getActionDependencies 35/head
Yaniv Rodenski [Sun, 21 Oct 2018 03:17:11 +0000 (14:17 +1100)] 
added getActionResources and getActionDependencies

7 weeks agoadded getActionResources and getActionDependencies
Yaniv Rodenski [Sun, 21 Oct 2018 00:54:14 +0000 (11:54 +1100)] 
added getActionResources and getActionDependencies

7 weeks agoAMATERASU-51: Add support for configuring the path for action level environment path
Eran Bartenstein [Wed, 17 Oct 2018 21:44:18 +0000 (08:44 +1100)] 
AMATERASU-51: Add support for configuring the path for action level environment path
Add gradle Kotlin support in Ama and more specifically in leader

8 weeks agoAmaterasu 49 - mesos implementation (#34)
Yaniv Rodenski [Sat, 13 Oct 2018 06:00:16 +0000 (17:00 +1100)] 
Amaterasu 49 - mesos implementation (#34)

* added ConfigManager + tests

* added support for custom path in actions

* moved tests to use assertEquals

* added support for framework configuration

* updated jeckson version

* env.yml file is created in mesos containers

* added runtime.yaml and datastores.yaml

* added actionName to runtime.yaml

* fixed git tests

* fixed newline in runtime.yaml

8 weeks agoremoved uneeded chars
Yaniv Rodenski [Fri, 12 Oct 2018 11:02:39 +0000 (22:02 +1100)] 
removed uneeded chars

4 months agoadded spark-shell provider
Yaniv Rodenski [Sun, 5 Aug 2018 10:57:34 +0000 (20:57 +1000)] 
added spark-shell provider

4 months agoMerge pull request #32 from roadan/AMATERASU-35
Yaniv Rodenski [Sun, 5 Aug 2018 09:38:26 +0000 (19:38 +1000)] 
Merge pull request #32 from roadan/AMATERASU-35

Amaterasu 35

4 months agoparameterized spark version in SparkSetupProvider.scala 32/head
Yaniv Rodenski [Wed, 1 Aug 2018 02:53:54 +0000 (12:53 +1000)] 
parameterized spark version in SparkSetupProvider.scala

4 months agobroke mesos cmd to multiline
Yaniv Rodenski [Tue, 31 Jul 2018 12:33:14 +0000 (22:33 +1000)] 
broke mesos cmd to multiline

4 months agoadded callback address to spark scala
Yaniv Rodenski [Mon, 30 Jul 2018 08:34:13 +0000 (18:34 +1000)] 
added callback address to spark scala

4 months agofixed mesos classpath
Yaniv Rodenski [Sat, 28 Jul 2018 06:36:57 +0000 (16:36 +1000)] 
fixed mesos classpath

4 months agospark runners providers added
Yaniv Rodenski [Thu, 26 Jul 2018 07:33:46 +0000 (17:33 +1000)] 
spark runners providers added

4 months agoMerge pull request #31 from roadan/AMATERASU-47
Eyal Ben Ivri [Tue, 24 Jul 2018 13:06:13 +0000 (15:06 +0200)] 
Merge pull request #31 from roadan/AMATERASU-47

Amaterasu 47

4 months agoincreased mem for yarn jobs 31/head
Yaniv Rodenski [Mon, 23 Jul 2018 10:00:32 +0000 (20:00 +1000)] 
increased mem for yarn jobs

4 months agofixed haddop classpath
Yaniv Rodenski [Sun, 22 Jul 2018 07:56:27 +0000 (17:56 +1000)] 
fixed haddop classpath

4 months agocreated spark-dispatcher and leader-common projects
Yaniv Rodenski [Tue, 17 Jul 2018 09:23:10 +0000 (19:23 +1000)] 
created spark-dispatcher and leader-common projects

4 months agoMerge remote-tracking branch 'upstream/master'
Yaniv Rodenski [Tue, 17 Jul 2018 02:38:49 +0000 (12:38 +1000)] 
Merge remote-tracking branch 'upstream/master'

4 months agoMerge pull request #25 from roadan/AMATERASU-33
Yaniv Rodenski [Tue, 17 Jul 2018 01:40:45 +0000 (11:40 +1000)] 
Merge pull request #25 from roadan/AMATERASU-33

Amaterasu 33

4 months agomerged with latest master 25/head
Yaniv Rodenski [Tue, 17 Jul 2018 01:28:27 +0000 (11:28 +1000)] 
merged with latest master

4 months agoMerge remote-tracking branch 'upstream/master' into AMATERASU-33
Yaniv Rodenski [Tue, 17 Jul 2018 01:01:18 +0000 (11:01 +1000)] 
Merge remote-tracking branch 'upstream/master' into AMATERASU-33

4 months agoMerge remote-tracking branch 'upstream/master'
Yaniv Rodenski [Mon, 16 Jul 2018 04:55:11 +0000 (14:55 +1000)] 
Merge remote-tracking branch 'upstream/master'

5 months agoMerge pull request #30 from arunma/Amaterasu-24-Master
Yaniv Rodenski [Sat, 7 Jul 2018 08:28:53 +0000 (18:28 +1000)] 
Merge pull request #30 from arunma/Amaterasu-24-Master

AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own pro…

5 months agofixed miniconda typo 30/head
Yaniv Rodenski [Sat, 7 Jul 2018 01:46:10 +0000 (11:46 +1000)] 
fixed miniconda typo

5 months agofixed mesos classpath
Yaniv Rodenski [Sat, 7 Jul 2018 01:13:56 +0000 (11:13 +1000)] 
fixed mesos classpath

5 months agoAMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project (Master)
Arun Manivannan [Sun, 1 Jul 2018 07:31:53 +0000 (15:31 +0800)] 
AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project (Master)

5 months agoMerge pull request #28 from apache/version-0.2.0-incubating-rc4
Yaniv Rodenski [Sun, 1 Jul 2018 02:24:16 +0000 (12:24 +1000)] 
Merge pull request #28 from apache/version-0.2.0-incubating-rc4

Version 0.2.0 incubating rc4

5 months agoMerge pull request #29 from apache/revert-27-RC4-RefactorFramework version-0.2.0-incubating-rc4 28/head
Yaniv Rodenski [Sun, 1 Jul 2018 00:21:47 +0000 (10:21 +1000)] 
Merge pull request #29 from apache/revert-27-RC4-RefactorFramework

Revert "AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project"

5 months agoRevert "AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project" revert-27-RC4-RefactorFramework 29/head
Yaniv Rodenski [Sun, 1 Jul 2018 00:18:46 +0000 (10:18 +1000)] 
Revert "AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project"

5 months agoMerge pull request #27 from arunma/RC4-RefactorFramework
Yaniv Rodenski [Sun, 1 Jul 2018 00:02:42 +0000 (10:02 +1000)] 
Merge pull request #27 from arunma/RC4-RefactorFramework

AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project

5 months agoAMATERASU-24 - Refactor Spark out of Amaterasu executor to it's own project 27/head
Arun Manivannan [Thu, 28 Jun 2018 05:37:14 +0000 (13:37 +0800)] 
AMATERASU-24 - Refactor Spark out of Amaterasu executor to it's own project

5 months agoAMATERASU-24 - Refactor Spark out of Amaterasu executor to it's own project
Arun Manivannan [Sun, 24 Jun 2018 09:24:21 +0000 (17:24 +0800)] 
AMATERASU-24 - Refactor Spark out of Amaterasu executor to it's own project

5 months agoAMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project
Arun Manivannan [Sat, 23 Jun 2018 16:43:07 +0000 (00:43 +0800)] 
AMATERASU-24 Refactor Spark out of Amaterasu executor to it's own project

5 months agoadding tags back totravis releases version-0.2.0-incubating-rc4
Yaniv Rodenski [Sat, 23 Jun 2018 00:46:12 +0000 (10:46 +1000)] 
adding tags back totravis releases

5 months agoremoved release tag
Yaniv Rodenski [Fri, 22 Jun 2018 00:43:53 +0000 (10:43 +1000)] 
removed release tag

5 months agoadded release commitish and tag
Yaniv Rodenski [Fri, 22 Jun 2018 00:14:06 +0000 (10:14 +1000)] 
added release commitish and tag

5 months agoadded missing licenses
Yaniv Rodenski [Thu, 21 Jun 2018 06:12:07 +0000 (16:12 +1000)] 
added missing licenses

5 months agoremoving the gradle-warpper.jar from source distribution
Yaniv Rodenski [Thu, 21 Jun 2018 05:48:54 +0000 (15:48 +1000)] 
removing the gradle-warpper.jar from source distribution

5 months agomerged from master support for multiline comments
Yaniv Rodenski [Sat, 9 Jun 2018 04:23:57 +0000 (14:23 +1000)] 
merged from master support for multiline comments

6 months agoupgraded gradle version to 4.8
Yaniv Rodenski [Sun, 10 Jun 2018 04:45:27 +0000 (14:45 +1000)] 
upgraded gradle version to 4.8

6 months agoMerge remote-tracking branch 'upstream/master'
Yaniv Rodenski [Sun, 10 Jun 2018 04:39:53 +0000 (14:39 +1000)] 
Merge remote-tracking branch 'upstream/master'

6 months agoadded checking for multiline comments in scala
Yaniv Rodenski [Sat, 9 Jun 2018 04:23:57 +0000 (14:23 +1000)] 
added checking for multiline comments in scala

6 months agoMerge branch 'master' of github.com:apache/incubator-amaterasu
Yaniv Rodenski [Sat, 9 Jun 2018 04:10:40 +0000 (14:10 +1000)] 
Merge branch 'master' of github.com:apache/incubator-amaterasu

6 months agoMerge pull request #23 from apache/version-0.2.0-incubating-rc3
Yaniv Rodenski [Sat, 9 Jun 2018 04:10:18 +0000 (14:10 +1000)] 
Merge pull request #23 from apache/version-0.2.0-incubating-rc3

Version 0.2.0 incubating rc3

6 months agoMerge branch 'version-0.2.0-incubating-rc3' of github.com:apache/incubator-amaterasu...
Yaniv Rodenski [Sat, 9 Jun 2018 04:08:08 +0000 (14:08 +1000)] 
Merge branch 'version-0.2.0-incubating-rc3' of github.com:apache/incubator-amaterasu into version-0.2.0-incubating-rc3

6 months agoadded checking for multiline comments in scala
Yaniv Rodenski [Sat, 9 Jun 2018 04:03:41 +0000 (14:03 +1000)] 
added checking for multiline comments in scala

6 months agoMerge tag 'version-0.2.0-incubating-rc3'
Yaniv Rodenski [Sat, 9 Jun 2018 03:56:18 +0000 (13:56 +1000)] 
Merge tag 'version-0.2.0-incubating-rc3'

6 months agoMerge branch 'master' into version-0.2.0-incubating-rc3 version-0.2.0-incubating-rc3 23/head
Yaniv Rodenski [Sat, 9 Jun 2018 03:47:49 +0000 (13:47 +1000)] 
Merge branch 'master' into version-0.2.0-incubating-rc3

6 months agoMerge pull request #20 from nadav-har-tzvi/version-0.2.0-incubating-rc3
Yaniv Rodenski [Mon, 28 May 2018 01:25:21 +0000 (11:25 +1000)] 
Merge pull request #20 from nadav-har-tzvi/version-0.2.0-incubating-rc3

PySpark fixes for YARN and Mesos

6 months agoapparently we need the previous config 20/head
Nadav Har Tzvi [Sun, 27 May 2018 06:43:13 +0000 (09:43 +0300)] 
apparently we need the previous config

6 months agofixing review comments
Nadav Har Tzvi [Sun, 27 May 2018 05:54:42 +0000 (08:54 +0300)] 
fixing review comments

6 months agoPySpark fixes for YARN and Mesos
Nadav Har Tzvi [Thu, 17 May 2018 06:00:24 +0000 (09:00 +0300)] 
PySpark fixes for YARN and Mesos
includes a hack for executor tests to run

6 months agoMerge pull request #19 from amezng/amaterasu-21 version-0.2.0-incubating-rc3
Eyal Ben Ivri [Sat, 26 May 2018 08:09:45 +0000 (10:09 +0200)] 
Merge pull request #19 from amezng/amaterasu-21

AMATERASU-21 Fix Spark scala tests

6 months agoMerge pull request #18 from arunma/version-0.2.0-incubating-rc3
Eyal Ben Ivri [Sat, 26 May 2018 07:54:40 +0000 (09:54 +0200)] 
Merge pull request #18 from arunma/version-0.2.0-incubating-rc3

AMATERASU-26 Pipeline tasks runs as "yarn" user instead of inheriting…

6 months agoFixed compilation error due to license file 19/head
Shad Amez [Tue, 22 May 2018 12:29:11 +0000 (17:59 +0530)] 
Fixed compilation error due to license file

6 months agoFixed SparkScalaRunnerTests failures
Shad Amez [Tue, 22 May 2018 12:15:26 +0000 (17:45 +0530)] 
Fixed SparkScalaRunnerTests failures

6 months agoAMATERASU-26 Pipeline tasks runs as "yarn" user instead of inheriting the user 18/head
Arun Manivannan [Fri, 18 May 2018 12:54:36 +0000 (20:54 +0800)] 
AMATERASU-26 Pipeline tasks runs as "yarn" user instead of inheriting the user

6 months agoadded 3rd party licenses
Yaniv Rodenski [Tue, 15 May 2018 08:19:48 +0000 (18:19 +1000)] 
added 3rd party licenses

6 months agofixed pyspark stest
Yaniv Rodenski [Tue, 15 May 2018 02:47:27 +0000 (12:47 +1000)] 
fixed pyspark stest

6 months agofixed missing licenses, Apache Rat runs successfuly
Yaniv Rodenski [Tue, 15 May 2018 02:21:52 +0000 (12:21 +1000)] 
fixed missing licenses, Apache Rat runs successfuly

6 months agofixed missing licenses, Apache Rat runs successfuly
Yaniv Rodenski [Tue, 15 May 2018 02:17:13 +0000 (12:17 +1000)] 
fixed missing licenses, Apache Rat runs successfuly

6 months agoMerge remote-tracking branch 'upstream/version-0.2.0-incubating-rc3'
Yaniv Rodenski [Tue, 15 May 2018 01:16:54 +0000 (11:16 +1000)] 
Merge remote-tracking branch 'upstream/version-0.2.0-incubating-rc3'

6 months agoremoved debug msgs
Yaniv Rodenski [Mon, 14 May 2018 11:10:05 +0000 (21:10 +1000)] 
removed debug msgs

6 months agodebug msgs
Yaniv Rodenski [Mon, 14 May 2018 10:13:41 +0000 (20:13 +1000)] 
debug msgs

6 months agofixed a spark path issue due to a bad merge
Yaniv Rodenski [Mon, 14 May 2018 08:46:56 +0000 (18:46 +1000)] 
fixed a spark path issue due to a bad merge

6 months agofixed missing license in files
Yaniv Rodenski [Mon, 14 May 2018 01:14:14 +0000 (11:14 +1000)] 
fixed missing license in files

7 months agoadded task to copy the leagal files
Yaniv Rodenski [Sun, 6 May 2018 11:20:13 +0000 (21:20 +1000)] 
added task to copy the leagal files

7 months agofixed framework distribution in clean env version-0.2.0-incubating-rc2 version-0.2.0-incubating-rc2
Yaniv Rodenski [Mon, 23 Apr 2018 04:38:53 +0000 (14:38 +1000)] 
fixed framework distribution in clean env

7 months agofixed merge issue
Yaniv Rodenski [Sun, 22 Apr 2018 08:56:40 +0000 (18:56 +1000)] 
fixed merge issue

7 months agoadded tests to travis
Yaniv Rodenski [Thu, 19 Apr 2018 23:57:31 +0000 (09:57 +1000)] 
added tests to travis

7 months agofixed build order issue
Yaniv Rodenski [Thu, 19 Apr 2018 11:50:32 +0000 (21:50 +1000)] 
fixed build order issue

7 months agofixed build order in build.gradle
Yaniv Rodenski [Thu, 19 Apr 2018 11:43:50 +0000 (21:43 +1000)] 
fixed build order in build.gradle