incubator-heron.git
2 weeks ago[Scala Streamlet API] Add Integration Test for Source and Union Operations (#3121) master
Eren Avsarogullari [Mon, 26 Nov 2018 22:50:27 +0000 (22:50 +0000)] 
[Scala Streamlet API] Add Integration Test for Source and Union Operations (#3121)

* [Scala Streamlet API] Add New Integration Test

* Documentation is added

2 weeks agoAdd withStream() in Streamlet to support stream selection (#3109)
Ning Wang [Mon, 26 Nov 2018 18:37:46 +0000 (10:37 -0800)] 
Add withStream() in Streamlet to support stream selection (#3109)

* Add withStream() in Streamlet to support stream selection

* Add split() function into Streamlet API

2 weeks ago[Java Streamlet API] Support Abstractions on Streamlet Sources (#3118)
Eren Avsarogullari [Mon, 26 Nov 2018 16:09:13 +0000 (16:09 +0000)] 
[Java Streamlet API] Support Abstractions on Streamlet Sources (#3118)

2 weeks agoSupport utils.Time of kafka-client-spout (#3116)
se choi [Thu, 22 Nov 2018 04:55:55 +0000 (13:55 +0900)] 
Support utils.Time of kafka-client-spout (#3116)

* Support utils.Time of kafka-client-spout

2 weeks agoNwang/add streamlet operator interfaces with grouping v2 (#3099)
Ning Wang [Thu, 22 Nov 2018 00:40:02 +0000 (16:40 -0800)] 
Nwang/add streamlet operator interfaces with grouping v2 (#3099)

* Add grouping support for Streamlet.applyOperator()

* Add support in Scala

* Simplify grouping support in Streamlet to allow only one incoming stream

3 weeks ago[Java Streamlet API] Move Source Logics to Builder (#3115)
Eren Avsarogullari [Wed, 21 Nov 2018 01:29:17 +0000 (01:29 +0000)] 
[Java Streamlet API] Move Source Logics to Builder (#3115)

3 weeks agoEnable Streamlet Scala API Documentation (#3114)
Eren Avsarogullari [Mon, 19 Nov 2018 23:53:02 +0000 (23:53 +0000)] 
Enable Streamlet Scala API Documentation (#3114)

3 weeks ago[Java Streamlet API] Support abstractions on Streamlet Operators (#3112)
Eren Avsarogullari [Mon, 19 Nov 2018 20:56:08 +0000 (20:56 +0000)] 
[Java Streamlet API] Support abstractions on Streamlet Operators (#3112)

* [Java Streamlet API] Support Abstractions on Streamlet Operators

3 weeks agoUpdate mailing list informtion in contributing.md (#3110)
Ning Wang [Mon, 19 Nov 2018 19:30:34 +0000 (11:30 -0800)] 
Update mailing list informtion in contributing.md (#3110)

3 weeks ago[Java Streamlet API] Extend Validations Part II (#3111)
Eren Avsarogullari [Mon, 19 Nov 2018 16:05:37 +0000 (16:05 +0000)] 
[Java Streamlet API] Extend Validations Part II (#3111)

* Extends Streamlet Java API Validations Part II

* Set otherStreamlet parameter name as more meaningful

4 weeks agoNwang/add spout based source streamlet (#3032)
Ning Wang [Fri, 9 Nov 2018 23:25:30 +0000 (15:25 -0800)] 
Nwang/add spout based source streamlet (#3032)

* Add spout based Streamlet source

4 weeks agofix typo (#3108)
qiao hai-jun [Fri, 9 Nov 2018 16:06:39 +0000 (00:06 +0800)] 
fix typo (#3108)

4 weeks agoPort Scala Streamlet integration tests to Java (#3105)
Ning Wang [Thu, 8 Nov 2018 22:52:39 +0000 (14:52 -0800)] 
Port Scala Streamlet integration tests to Java (#3105)

5 weeks ago[Java Streamlet API] Extend validations (#3095)
Eren Avsarogullari [Mon, 5 Nov 2018 19:40:08 +0000 (19:40 +0000)] 
[Java Streamlet API] Extend validations (#3095)

* Add validations to Streamlet API

* Review comments are addressed.

5 weeks agoFix build error on Mac. (#3102)
se choi [Mon, 5 Nov 2018 19:39:39 +0000 (04:39 +0900)] 
Fix build error on Mac. (#3102)

* Fix build error on Mac.

* Change equalsIgnoreCase to equals

5 weeks agoAdd support for building Heron release in a Ubuntu 18.04 docker container (#3097)
Thomas Cooper [Sat, 3 Nov 2018 03:02:15 +0000 (03:02 +0000)] 
Add support for building Heron release in a Ubuntu 18.04 docker container (#3097)

* Added ubuntu 18.04 compile and dist dockerfiles

* Updated script help text with supported distributions

* Removed defunct run heron in docker instructions

5 weeks agoNwang/refactor custom operator interface v2 (#3100)
Ning Wang [Sat, 3 Nov 2018 03:01:05 +0000 (20:01 -0700)] 
Nwang/refactor custom operator interface v2 (#3100)

* Reduce number of applyOperator() functions in Streamlet

* Remove CustomBasicStreamlet and CustomWindowStreamlet

5 weeks agoFix number of tasks in Streamlet RemapCustomGrouping (#3098)
Ning Wang [Fri, 2 Nov 2018 20:45:40 +0000 (13:45 -0700)] 
Fix number of tasks in Streamlet RemapCustomGrouping (#3098)

6 weeks agoUpdate pidhandler.py (#3094)
Windham Wong [Wed, 31 Oct 2018 05:58:07 +0000 (13:58 +0800)] 
Update pidhandler.py (#3094)

Fixing Bash syntax error in pid checking script.

6 weeks agoenablebool (#3056)
bed debug [Fri, 26 Oct 2018 23:49:07 +0000 (16:49 -0700)] 
enablebool (#3056)

6 weeks agoAdd interfaces IStreamletOperator, IStreamletBasicOperator and IStrea… (#3051)
Ning Wang [Wed, 24 Oct 2018 18:27:57 +0000 (11:27 -0700)] 
Add interfaces IStreamletOperator, IStreamletBasicOperator and IStrea… (#3051)

* Add interfaces IStreamletOperator, IStreamletBasicOperator and IStreamletWindowOperator and support them in Streamlet

* Fix HiddenField

7 weeks agoUpdate S3 uploader documentations (#3053)
ryan-dyer-sp [Tue, 23 Oct 2018 17:35:24 +0000 (12:35 -0500)] 
Update S3 uploader documentations (#3053)

* Update s3 uploader documentation

* correct var name

* update brew installation command docs

7 weeks agoadd back stmgr metrics (#3052)
Yao Li [Mon, 22 Oct 2018 21:55:04 +0000 (14:55 -0700)] 
add back stmgr metrics (#3052)

* add back stmgr metrics

* refactor and add comments on stmgr-server metrics registration

7 weeks agoNwang/refactor grouping (#3040)
Ning Wang [Fri, 19 Oct 2018 21:35:26 +0000 (14:35 -0700)] 
Nwang/refactor grouping (#3040)

* Refactor grouping in BoltDeclarer

* clean up

* fix direct grouping

* clean up

2 months agoFix annotation processor (#3038) release/v-0.20.0-incubating 3084/head v-0.20.0-incubating v-0.20.0-incubating-candidate-5
Ning Wang [Thu, 11 Oct 2018 02:25:40 +0000 (19:25 -0700)] 
Fix annotation processor (#3038)

* Fix annotation processor

Comments is not allowed. Got rid of the license info and exluce it from
rat

* clean up

2 months agoRefactor heron_executor to make it easier to customize (#3043)
Ning Wang [Wed, 10 Oct 2018 20:57:53 +0000 (13:57 -0700)] 
Refactor heron_executor to make it easier to customize (#3043)

* Refactor heron_executor to make it easier to customize

* fix unittest

2 months agoRemove libunwind patches (#3041)
Ning Wang [Wed, 10 Oct 2018 17:37:06 +0000 (10:37 -0700)] 
Remove libunwind patches (#3041)

* Remove libunwind patches

* disable minidebuginfo

* Clean up

2 months agocleanup README (#3037)
Neng Lu [Tue, 9 Oct 2018 20:56:38 +0000 (13:56 -0700)] 
cleanup README (#3037)

2 months agoFix Topology-API window documentation (#3039)
Eren Avsarogullari [Tue, 9 Oct 2018 20:56:16 +0000 (21:56 +0100)] 
Fix Topology-API window documentation (#3039)

2 months agoNwang/update thirdparty license (#3042)
Ning Wang [Tue, 9 Oct 2018 20:54:18 +0000 (13:54 -0700)] 
Nwang/update thirdparty license (#3042)

* Update thrift party licenses in LICENSE

* clean up

2 months agoFix 'Copyright 2015 The Heron Authors' in scripts (#3036)
Ning Wang [Fri, 28 Sep 2018 15:07:17 +0000 (08:07 -0700)] 
Fix 'Copyright 2015 The Heron Authors' in scripts (#3036)

2 months agoRemove unnecessary createSupplierStreamlet from Scala StreamletImpl (#3030)
Ning Wang [Tue, 25 Sep 2018 23:18:06 +0000 (16:18 -0700)] 
Remove unnecessary createSupplierStreamlet from Scala StreamletImpl (#3030)

2 months agoNwang/remove stateful from streamletoperator (#3034)
Ning Wang [Tue, 25 Sep 2018 23:17:12 +0000 (16:17 -0700)] 
Nwang/remove stateful from streamletoperator (#3034)

* Refactor stateful support into operator implementations from base class

* clean up

* clean up unused imports

* clean up TODO

2 months agofix to generate correct artificats [https://github.com/apache/incubat… (#3014) v-0.20.0-incubating-candidate-4
SreeV [Mon, 24 Sep 2018 07:48:58 +0000 (00:48 -0700)] 
fix to generate correct artificats [https://github.com/apache/incubat… (#3014)

* fix to generate correct artificats [https://github.com/apache/incubator-heron/issues/3013] - Sree Vaddi

* fixed links, updated text [https://github.com/apache/incubator-heron/issues/3015] - Sree Vaddi

2 months agorefactor JavaSerializer for proper closing input/output streams (#3031)
Neng Lu [Thu, 20 Sep 2018 19:02:44 +0000 (12:02 -0700)] 
refactor JavaSerializer for proper closing input/output streams (#3031)

2 months agoRevert "refactor JavaSerializer for closing input/output stream"
Neng Lu [Wed, 19 Sep 2018 22:00:57 +0000 (15:00 -0700)] 
Revert "refactor JavaSerializer for closing input/output stream"

This reverts commit db3f1c406c7ff73113e2c469b44636baee1eb77f.

2 months agorefactor JavaSerializer for closing input/output stream
Neng Lu [Wed, 19 Sep 2018 21:56:56 +0000 (14:56 -0700)] 
refactor JavaSerializer for closing input/output stream

2 months agoheron shell access directory
Neng Lu [Tue, 18 Sep 2018 18:58:21 +0000 (11:58 -0700)] 
heron shell access directory

2 months agoAdd license headers and rat-exception file (#3011)
Ning Wang [Tue, 18 Sep 2018 06:17:46 +0000 (23:17 -0700)] 
Add license headers and rat-exception file (#3011)

2 months agofix integration test failure with topo parallelism update (#3026)
Yao Li [Fri, 14 Sep 2018 21:12:55 +0000 (14:12 -0700)] 
fix integration test failure with topo parallelism update (#3026)

* fix integration test failure with topo parallelism update

* fix typo

2 months agoadd max thread limit in integration test (#3025)
Yao Li [Fri, 14 Sep 2018 08:46:04 +0000 (01:46 -0700)] 
add max thread limit in integration test (#3025)

merge this pr for integration threads fix. the 3 integration failure fix will be in the next pr

2 months agorun integration tests parallelly (#3024)
Yao Li [Wed, 12 Sep 2018 18:45:24 +0000 (11:45 -0700)] 
run integration tests parallelly (#3024)

* run integration tests parallelly

* fix bugs in multi-threading.

3 months agouse RoundRobinPacking algorithm by default (#3022)
Neng Lu [Tue, 11 Sep 2018 23:57:32 +0000 (16:57 -0700)] 
use RoundRobinPacking algorithm by default (#3022)

* use RoundRobinPacking algorithm by default

* update repacking algorithm

3 months agofixstyle (#3023)
bed debug [Tue, 11 Sep 2018 22:56:36 +0000 (15:56 -0700)] 
fixstyle (#3023)

3 months agoFix build of eco/tests/... and scripts/release/... (#3012)
Ning Wang [Tue, 11 Sep 2018 22:43:12 +0000 (15:43 -0700)] 
Fix build of eco/tests/... and scripts/release/... (#3012)

3 months agoLog executor command line arguments (#3020)
Ning Wang [Thu, 6 Sep 2018 17:46:26 +0000 (10:46 -0700)] 
Log executor command line arguments (#3020)

3 months agofix healthmgr unit tests (#3018)
bed debug [Tue, 4 Sep 2018 23:09:13 +0000 (16:09 -0700)] 
fix healthmgr unit tests (#3018)

* addBUILDforhealthmgr

* fixhealthmgrunit

3 months agoUpdate README(updated urls and added developer link) (#3010)
Ning Wang [Tue, 4 Sep 2018 17:38:24 +0000 (10:38 -0700)] 
Update README(updated urls and added developer link) (#3010)

3 months agoFix javadoc errors (#3017)
Ning Wang [Tue, 4 Sep 2018 17:28:59 +0000 (10:28 -0700)] 
Fix javadoc errors (#3017)

3 months agorevert heron_excutor.py - delete unnecessary logs (#3008)
Yao Li [Mon, 27 Aug 2018 20:25:41 +0000 (13:25 -0700)] 
revert heron_excutor.py - delete unnecessary logs (#3008)

3 months agoFix path and package name in docs/comments/tests (#3007)
Ning Wang [Fri, 24 Aug 2018 17:26:16 +0000 (10:26 -0700)] 
Fix path and package name in docs/comments/tests (#3007)

3 months agoYaoli/fix stateful test (#3004)
Yao Li [Fri, 24 Aug 2018 07:50:03 +0000 (00:50 -0700)] 
Yaoli/fix stateful test (#3004)

* fix duplicated state info at http server

* deleted unused import

3 months agoFix RAM config in example code (#3006) v-0.20.0-incubating-candidate-3
Ning Wang [Thu, 23 Aug 2018 20:05:19 +0000 (13:05 -0700)] 
Fix RAM config in example code (#3006)

3 months agofix building problem with source tarball (#3005)
Neng Lu [Thu, 23 Aug 2018 18:21:01 +0000 (11:21 -0700)] 
fix building problem with source tarball (#3005)

* fix status.sh without .git info

3 months agoRemove icomoon usage from tools/ui (#2999) v-0.20.0-incubating-candidate-2
Ning Wang [Wed, 22 Aug 2018 17:02:48 +0000 (10:02 -0700)] 
Remove icomoon usage from tools/ui (#2999)

3 months agoMove license header in react dom components (#3003)
Ning Wang [Wed, 22 Aug 2018 17:02:17 +0000 (10:02 -0700)] 
Move license header in react dom components (#3003)

* Move license header in react dom components

* remove empty line at beginning

3 months agochmodcicheck (#3002)
bed debug [Wed, 22 Aug 2018 03:18:29 +0000 (20:18 -0700)] 
chmodcicheck (#3002)

3 months agofixunit (#3001)
bed debug [Wed, 22 Aug 2018 01:17:45 +0000 (18:17 -0700)] 
fixunit (#3001)

3 months agofixunit (#3000)
bed debug [Tue, 21 Aug 2018 23:42:19 +0000 (16:42 -0700)] 
fixunit (#3000)

3 months agoenablebypassintegrationtests (#2997)
bed debug [Tue, 21 Aug 2018 22:27:43 +0000 (15:27 -0700)] 
enablebypassintegrationtests (#2997)

3 months agoFix the handling of container number in api server (#2995)
Ning Wang [Tue, 21 Aug 2018 22:27:22 +0000 (15:27 -0700)] 
Fix the handling of container number in api server (#2995)

3 months agoNwang/make spout/bolt compatible to storm 1.x (#2993)
Ning Wang [Tue, 21 Aug 2018 22:27:11 +0000 (15:27 -0700)] 
Nwang/make spout/bolt compatible to storm 1.x (#2993)

* Make storm-compatible api to be compatible with storm 1.x

* more files

* Fix in eco-storm-examples

3 months agoget back newParallelism ifelse branch (#2992)
bed debug [Tue, 21 Aug 2018 22:26:58 +0000 (15:26 -0700)] 
get back newParallelism ifelse branch (#2992)

* get back newParallelism ifelse branch

* revertunit test

* reusecode

* reorg

* mergelog

3 months agoAdd gitattributes to remove website directory from archive (#2996)
Ning Wang [Tue, 21 Aug 2018 19:05:36 +0000 (12:05 -0700)] 
Add gitattributes to remove website directory from archive (#2996)

3 months agoremove crreal BUILD file (#2988)
Neng Lu [Tue, 21 Aug 2018 04:10:50 +0000 (21:10 -0700)] 
remove crreal BUILD file (#2988)

3 months agoChmod travis ci.sh (#2991)
Ning Wang [Mon, 20 Aug 2018 23:37:49 +0000 (16:37 -0700)] 
Chmod travis ci.sh (#2991)

* Chmod travis ci.sh

* Add bazel_config.py

3 months agoRevert mode change in sh files (#2990)
Ning Wang [Mon, 20 Aug 2018 22:45:48 +0000 (15:45 -0700)] 
Revert mode change in sh files (#2990)

3 months agoNwang/clean trailing white space (#2989)
Ning Wang [Mon, 20 Aug 2018 22:30:21 +0000 (15:30 -0700)] 
Nwang/clean trailing white space (#2989)

* Clean up trailing spaces

* update

3 months agoYaoli/test (#2987)
Yao Li [Mon, 20 Aug 2018 20:29:53 +0000 (13:29 -0700)] 
Yaoli/test (#2987)

* just test

* only test the second one

* add log in heron_executor.py

* reset test.json

* fix test.json

3 months agouse cereal lib configured in WORKSPACE (#2986)
Ning Wang [Mon, 20 Aug 2018 17:27:21 +0000 (10:27 -0700)] 
use cereal lib configured in WORKSPACE (#2986)

3 months agoExtending update command to allow number of containers to be modified at runtime...
Faria Kalim [Fri, 17 Aug 2018 15:38:24 +0000 (08:38 -0700)] 
Extending update command to allow number of containers to be modified at runtime (#2980)

* added num containers to heron cli

* bug fix

* updated apiserver to accept container num

* removed bug that did allow correct parallelism to be supplied when new containers were supplied

* corrected variable in update.py

* bug fix

* removed unused comment

* added cumulative outgoing queue size

* added num containers to heron cli

* bug fix

* updated apiserver to accept container num

* removed bug that did allow correct parallelism to be supplied when new containers were supplied

* corrected variable in update.py

* bug fix

* removed unused comment

* added cumulative outgoing queue size

* bug fix

* remove extra code

* removed extra code

3 months agoMetrics added to measure the rate at which tuples are added to the queue at spout...
Faria Kalim [Fri, 17 Aug 2018 15:37:51 +0000 (08:37 -0700)] 
Metrics added to measure the rate at which tuples are added to the queue at spout gateway (#2981)

* added metrics to measure tuples added to outgoing queue per min

* added metric to full metrics instance

* added cumulative outgoing queue size

* measuring number of tuples per tupleset

* added comment justifying use of metrics

* removed extra comment

3 months agoNwang/add missing licenses (#2984)
Ning Wang [Fri, 17 Aug 2018 15:33:33 +0000 (08:33 -0700)] 
Nwang/add missing licenses (#2984)

* Add missing license headers

* revert non-heron files

* more files

3 months agoDocker Ubuntu tweaks (#2975) v-0.20.0-incubating-candidate-1
Thomas Cooper [Tue, 14 Aug 2018 18:46:32 +0000 (19:46 +0100)] 
Docker Ubuntu tweaks (#2975)

* Added .vagrant to gitignore

* Updated ubuntu docker images to use headless jdk + reduced update calls

- Altered the ubuntu dockerfiles to use the headless jdk, which prevents
the containers downloading a lot of pointless GUI libraries.

- Changed the command order of the 14.04 compile file to reduce the
overall number of `apt-get update` commands.

- Removed ubutnu 15.10 option as this version of ubuntu is no longer
supported with security updates etc.

- Updated docker/scripts help text to reflect the above

* Updated dockerfile and scripts for debian to v9 and remove apt-get clean

- Updated all references from debian 8 to 9 (as the Java slim images
uses debian 9).

- Updated all help text to refer to debian9.

- Removed apt-get clean commands from ubuntu docker files as they are
not required.

3 months agoupdate license information (#2977)
Neng Lu [Tue, 14 Aug 2018 18:01:46 +0000 (11:01 -0700)] 
update license information (#2977)

* update license information

* update JQuery License

3 months ago[healthmgr] fix style (#2983)
bed debug [Tue, 14 Aug 2018 01:53:54 +0000 (18:53 -0700)] 
[healthmgr] fix style (#2983)

* fixsimportimport

* fixstyle

3 months agofixsimportimport (#2982)
bed debug [Mon, 13 Aug 2018 23:25:41 +0000 (16:25 -0700)] 
fixsimportimport (#2982)

3 months ago[healthmgr] refactor physicalplanprovider and topologyprovider (#2979)
bed debug [Mon, 13 Aug 2018 22:45:00 +0000 (15:45 -0700)] 
[healthmgr] refactor physicalplanprovider and topologyprovider (#2979)

* verbose log for debug

* refactor healthmgr

* refactor 2

* refactor ppprovider

* fix compile

* fix pplan exception

* update print

* fixcomments

* change log to fine, when the verbose is enabled

* add verbose

* use list

4 months agoAdding average Serialization/Deserialization time per tuple (#2976)
Faria Kalim [Thu, 9 Aug 2018 20:29:39 +0000 (13:29 -0700)] 
Adding average Serialization/Deserialization time per tuple (#2976)

* adding metric to measure average serialization and deserialization times

* adding a metric that measures average serialization and deserialization time per tuple

4 months agoUpdate error message in RuntimeManagerMain when an exception is catched (#2974)
Ning Wang [Wed, 8 Aug 2018 17:15:03 +0000 (10:15 -0700)] 
Update error message in RuntimeManagerMain when an exception is catched (#2974)

4 months agoFixed Spelling. (#2971)
Jimmy Casey [Mon, 30 Jul 2018 16:18:54 +0000 (17:18 +0100)] 
Fixed Spelling. (#2971)

4 months agopackage c++ .h and .a files (#2961)
Karthik Ramasamy [Sat, 28 Jul 2018 06:48:53 +0000 (23:48 -0700)] 
package c++ .h and .a files (#2961)

* package c++ .h and cpp files

* remove unuwanted files

4 months agospill state to local disk when configured (#2967)
Neng Lu [Fri, 27 Jul 2018 21:31:53 +0000 (14:31 -0700)] 
spill state to local disk when configured (#2967)

* spill state to local disk when configured

* start ckptmgr-client after receiving pplan

* separate instances and ckptmgr spilled location

* clean states by owner

* bail out if spill state failed

* typo

4 months agofix metrics name (#2970)
bed debug [Sun, 22 Jul 2018 08:32:41 +0000 (01:32 -0700)] 
fix metrics name (#2970)

4 months agofixing download issue in Nomad scheduler (#2969)
Boyang Jerry Peng [Fri, 20 Jul 2018 19:05:59 +0000 (12:05 -0700)] 
fixing download issue in Nomad scheduler (#2969)

* fixing download issue in Nomad scheduler

* fix formatting

4 months agoUpgrade to the latest Dhalion (#2968)
Ashvin [Fri, 20 Jul 2018 13:36:24 +0000 (06:36 -0700)] 
Upgrade to the latest Dhalion (#2968)

* Upgrade to the latest Dhalion

Fixes #2927

* Remove unused import to fix checkstyle error

4 months ago(Ok To Merge) updating docker scripts (#2963)
Josh Fischer [Tue, 17 Jul 2018 21:20:50 +0000 (16:20 -0500)] 
(Ok To Merge) updating docker scripts (#2963)

* updating ubuntu14.04 docker scripts

* clean up

* updating other distros

* adding whitespace

4 months agoFix supervisor installation in Centos 7 Dockerfile (#2966)
Thomas Shields [Tue, 17 Jul 2018 21:18:51 +0000 (17:18 -0400)] 
Fix supervisor installation in Centos 7 Dockerfile (#2966)

4 months agoadd integration test for instance state in stateful processing (#2956)
Yao Li [Sat, 14 Jul 2018 00:09:56 +0000 (17:09 -0700)] 
add integration test for instance state in stateful processing (#2956)

* add integration test for instance state in stateful processing

* merge http servers

* remove the I prefix

5 months agojoshfischer/update k8s docs (#2960)
Josh Fischer [Thu, 12 Jul 2018 16:48:09 +0000 (11:48 -0500)] 
joshfischer/update k8s docs (#2960)

* change api status url

* further clean up on updated kuberntes versioned urls

* updating missed urls

* further clean up and addig disclaimers

* updating tool template with correct proxy url

* updating notes txt

5 months agoCounting pending tuples (#2951)
Faria Kalim [Thu, 12 Jul 2018 00:32:31 +0000 (17:32 -0700)] 
Counting pending tuples (#2951)

* only using tuple num in tuplestream message to count number of data tuples

* adding a field to tuplestreammessage to hold number of tuples in message

* removing redundant code line

5 months agoMake checkpoint manager's ram configurable (#2958)
Neng Lu [Wed, 11 Jul 2018 06:28:32 +0000 (23:28 -0700)] 
Make checkpoint manager's ram configurable (#2958)

* calculate container padding ram and make ckptmgr configurable

* add ckptmgr ram to aurora conf

* fix unit tests

5 months agoAdded endpoint to tracker to fetch packing plan. (#2959)
Faria Kalim [Tue, 10 Jul 2018 22:24:14 +0000 (15:24 -0700)] 
Added endpoint to tracker to fetch packing plan. (#2959)

* Added endpoint to tracker to fetch packing plan.

* checked scheduled resources as part of test

5 months ago[aurora] add metadata support for aurora (#2934)
bed debug [Sat, 7 Jul 2018 23:13:34 +0000 (16:13 -0700)] 
[aurora] add metadata support for aurora (#2934)

* add metadata field

* update aurora_metadata

* use 3 params

* fix unit

* fix comment

* update to Ning comment

* remove _x

* fix unit

* fix style

* split aurora config

* update filename

* fix idx

5 months agoRefactor StatefulStorage API (#2891)
Ning Wang [Sat, 7 Jul 2018 18:23:16 +0000 (11:23 -0700)] 
Refactor StatefulStorage API (#2891)

* Refactor StatefulStorage

* fix test

* fix CheckpointManagerServerTest

* component metadata is optional

* Update names to be more consistent

* remove data_version and partition_id from protobuf for now

* Remove metadata field from save/get checkpoint requests

* More name and comment changes

* revert refactor changes in CheckpointManagerServer

* Rename InstanceStateCheckpointPartition to InstanceStateCheckpoint

5 months agoAdd log in Bolt/Spout instances after persisting state (#2954)
Ning Wang [Sat, 7 Jul 2018 05:49:17 +0000 (22:49 -0700)] 
Add log in Bolt/Spout instances after persisting state (#2954)

5 months agoNwang/fix negative index caused by hash mod (#2953)
Ning Wang [Fri, 6 Jul 2018 21:42:00 +0000 (14:42 -0700)] 
Nwang/fix negative index caused by hash mod (#2953)

* Fix out of boundary issue caused by .hashCode() % taskIds.size()

* update RemapCustomGrouping

* Refactor and add unit test

5 months agoCounting pending tuples (#2945)
Faria Kalim [Tue, 3 Jul 2018 20:19:28 +0000 (13:19 -0700)] 
Counting pending tuples (#2945)

* added metric for counting num packets -- needs fixing

* added a map to hold task id -> task name mapping for help in mapping metrics

* fixing formatting and simplifying code

* ensuring metric exists before updatingit

5 months agoRevert the latest version number change of scala rules (#2943)
Ning Wang [Tue, 3 Jul 2018 18:47:38 +0000 (11:47 -0700)] 
Revert the latest version number change of scala rules (#2943)

* Revert the latest change about scala rules

* Revert the latest version number change of scala rules

5 months ago[healthmgr] Enable runtime toggle for AutoRestartBackpressureContainerPolicy (#2923)
bed debug [Tue, 3 Jul 2018 18:43:53 +0000 (11:43 -0700)] 
[healthmgr] Enable runtime toggle for AutoRestartBackpressureContainerPolicy (#2923)

* enable runtime config for autobackpressure policy

* update comment

* clean comment

* fix neng comment

* fix compile

* staging

* toggle each policy

* toggle per policy

* fix neng comment

* update condition check in toggle policy

* update log fine

* fix ning comment