incubator-sdap-ningester.git
7 weeks agoUpdate version (#13) master
fgreg [Thu, 18 Oct 2018 00:21:31 +0000 (17:21 -0700)] 
Update version (#13)

2 months agoSimple issue of not using the exit code returned from the spring app. (#12)
fgreg [Tue, 11 Sep 2018 22:31:51 +0000 (15:31 -0700)] 
Simple issue of not using the exit code returned from the spring app. (#12)

3 months agoUpdated docker image to include RANDOMIZE_NINGESTERPY_PORT environment variable....
fgreg [Wed, 29 Aug 2018 16:33:53 +0000 (09:33 -0700)] 
Updated docker image to include RANDOMIZE_NINGESTERPY_PORT environment variable. If this variable is set to anything but "false", the port that ningesterpy runs on will be a random available port. (#11)

3 months agoSDAP-111 Ningester support download from S3 (#10)
fgreg [Sat, 25 Aug 2018 00:07:50 +0000 (17:07 -0700)] 
SDAP-111 Ningester support download from S3 (#10)

* New image that can download granule from S3 before running

* New image that can download config from S3 before running

5 months agoSDAP-107 Ningester container does not report exit code correctly
fgreg [Thu, 14 Jun 2018 16:35:01 +0000 (09:35 -0700)] 
SDAP-107 Ningester container does not report exit code correctly

Now using `set -o pipefail` in entrypoint script.

7 months agoSDAP-65 ningesterpy output not printing in docker logs (#8)
fgreg [Thu, 26 Apr 2018 21:34:32 +0000 (14:34 -0700)] 
SDAP-65 ningesterpy output not printing in docker logs (#8)

* Add -u flag to python command for unbuffered stdout.

* Added wait for ningesterpy to start. Used -e and single quotes for sed.

* Made stdout not buffer with `stdbuf -o`

7 months agoSDAP-62 Fix No bean named 'pythonChainProcessor' available bug (#7)
fgreg [Wed, 18 Apr 2018 16:57:55 +0000 (09:57 -0700)] 
SDAP-62 Fix No bean named 'pythonChainProcessor' available bug  (#7)

Use the same `@ConditionalOnProperty` for processors in both `AppConfig` and `BatchConfig`

7 months agoSDAP-61 Added a test job for climatology ingestion.
fgreg [Tue, 17 Apr 2018 22:55:45 +0000 (15:55 -0700)] 
SDAP-61 Added a test job for climatology ingestion.

7 months agoSDAP-54 Added gcc and downgraded miniconda version to fix error with build.
fgreg [Tue, 17 Apr 2018 17:50:37 +0000 (10:50 -0700)] 
SDAP-54 Added gcc and downgraded miniconda version to fix error with build.

7 months agoSDAP-60 Added MySQL JDBC drivers to compile classpath.
fgreg [Tue, 17 Apr 2018 17:48:34 +0000 (10:48 -0700)] 
SDAP-60 Added MySQL JDBC drivers to compile classpath.

8 months agoSDAP-28 Ingest SWOT Sample Data Tiled by time (#2)
fgreg [Thu, 29 Mar 2018 18:29:04 +0000 (11:29 -0700)] 
SDAP-28 Ingest SWOT Sample Data Tiled by time (#2)

Added new tiling scheme that simply lets you set the step size for each dimension. Can be found at `src/main/java/org/apache/sdap/ningester/datatiler/SliceFileByStepSize.java`

10 months agoMerge pull request #1 from fgreg/master
fgreg [Tue, 30 Jan 2018 21:52:26 +0000 (13:52 -0800)] 
Merge pull request #1 from fgreg/master

SDAP-17: Working version of Ningester

10 months agoadded license headers 1/head
Frank Greguska [Tue, 30 Jan 2018 20:14:37 +0000 (12:14 -0800)] 
added license headers

10 months agofixed entrypoint command
Frank Greguska [Tue, 30 Jan 2018 01:06:15 +0000 (17:06 -0800)] 
fixed entrypoint command

10 months agoupdated docs and simplified config
Frank Greguska [Tue, 30 Jan 2018 00:29:23 +0000 (16:29 -0800)] 
updated docs and simplified config

10 months agoupdated test job
Frank Greguska [Sun, 28 Jan 2018 22:00:27 +0000 (14:00 -0800)] 
updated test job

10 months agofixed problem with the order beans were getting loaded
Frank Greguska [Sun, 28 Jan 2018 20:24:34 +0000 (12:24 -0800)] 
fixed problem with the order beans were getting loaded

10 months agolaunching ningesterpy process
Frank Greguska [Sat, 27 Jan 2018 03:16:25 +0000 (19:16 -0800)] 
launching ningesterpy process

10 months agoalmost working docker image
Frank Greguska [Sat, 27 Jan 2018 03:06:35 +0000 (19:06 -0800)] 
almost working docker image

10 months agoordering of templates was messed up
Frank Greguska [Sat, 27 Jan 2018 02:21:38 +0000 (18:21 -0800)] 
ordering of templates was messed up

10 months agoworking on docker install
Frank Greguska [Sat, 27 Jan 2018 02:03:45 +0000 (18:03 -0800)] 
working on docker install

10 months agoupdated ignore file
Frank Greguska [Tue, 23 Jan 2018 16:46:43 +0000 (08:46 -0800)] 
updated ignore file

10 months agoadd disclaimer update gradle version
Frank Greguska [Tue, 23 Jan 2018 02:29:05 +0000 (18:29 -0800)] 
add disclaimer update gradle version

10 months agogetting ready for Apache commit
Frank Greguska [Tue, 23 Jan 2018 00:16:23 +0000 (16:16 -0800)] 
getting ready for Apache commit

10 months agoupdated dependency to the new org.apache.sdap nexusproto project
Frank Greguska [Mon, 22 Jan 2018 23:41:19 +0000 (15:41 -0800)] 
updated dependency to the new org.apache.sdap nexusproto project

10 months agoupdated application to stop when job completes
Frank Greguska [Thu, 18 Jan 2018 21:51:44 +0000 (13:51 -0800)] 
updated application to stop when job completes

10 months agoRemoved batch configuration from AppConfig. Organized imports
Frank Greguska [Thu, 18 Jan 2018 01:46:23 +0000 (17:46 -0800)] 
Removed batch configuration from AppConfig. Organized imports

11 months agoadded a few more tests
Frank Greguska [Wed, 10 Jan 2018 00:28:29 +0000 (16:28 -0800)] 
added a few more tests

11 months agogot integration tests working
Frank Greguska [Wed, 10 Jan 2018 00:09:50 +0000 (16:09 -0800)] 
got integration tests working

11 months agocleaned up buildscript and split tests into unit tests and test jobs
Frank Greguska [Tue, 9 Jan 2018 18:23:24 +0000 (10:23 -0800)] 
cleaned up buildscript and split tests into unit tests and test jobs

11 months agowired up the writer and updated tests
Frank Greguska [Tue, 9 Jan 2018 17:40:56 +0000 (09:40 -0800)] 
wired up the writer and updated tests

11 months agoadded other datastores and their properties
Frank Greguska [Tue, 9 Jan 2018 02:05:39 +0000 (18:05 -0800)] 
added other datastores and their properties

11 months agoconsistent test names
Frank Greguska [Fri, 5 Jan 2018 01:33:15 +0000 (17:33 -0800)] 
consistent test names

11 months agogot solrstore and cassandra store moved over
Frank Greguska [Fri, 5 Jan 2018 01:29:46 +0000 (17:29 -0800)] 
got solrstore and cassandra store moved over

11 months agowired in the slicebydimension properties and added smap testjob
Frank Greguska [Thu, 4 Jan 2018 23:40:08 +0000 (15:40 -0800)] 
wired in the slicebydimension properties and added smap testjob

11 months agoadded slice by dimension slicer and test
Frank Greguska [Thu, 4 Jan 2018 22:59:03 +0000 (14:59 -0800)] 
added slice by dimension slicer and test

11 months agowired up the reader and changed it to produce NexusTiles instead of Strings
Frank Greguska [Thu, 4 Jan 2018 20:33:48 +0000 (12:33 -0800)] 
wired up the reader and changed it to produce NexusTiles instead of Strings

11 months agotrying out some new configuration properties
Frank Greguska [Wed, 3 Jan 2018 20:58:02 +0000 (12:58 -0800)] 
trying out some new configuration properties

11 months agoadded property files for each processor
Frank Greguska [Wed, 3 Jan 2018 20:19:07 +0000 (12:19 -0800)] 
added property files for each processor

11 months agoset dataset name processor with test
Frank Greguska [Wed, 3 Jan 2018 00:25:19 +0000 (16:25 -0800)] 
set dataset name processor with test

11 months agogenerate tile id processor added with tests
Frank Greguska [Wed, 3 Jan 2018 00:18:35 +0000 (16:18 -0800)] 
generate tile id processor added with tests

11 months agonew processor for adding time from granule name. updated tests
Frank Greguska [Tue, 2 Jan 2018 23:42:37 +0000 (15:42 -0800)] 
new processor for adding time from granule name. updated tests

11 months agonew processors and tests
Frank Greguska [Tue, 2 Jan 2018 23:06:18 +0000 (15:06 -0800)] 
new processors and tests

11 months agomore progress on batch job
Frank Greguska [Wed, 27 Dec 2017 22:32:16 +0000 (16:32 -0600)] 
more progress on batch job

13 months agoupdated reader and tests
Frank Greguska [Tue, 7 Nov 2017 23:00:04 +0000 (15:00 -0800)] 
updated reader and tests

13 months agoexperimenting with new Jython
Frank Greguska [Tue, 17 Oct 2017 16:21:57 +0000 (09:21 -0700)] 
experimenting with new Jython

13 months agoexperimenting with new Jython
Frank Greguska [Tue, 17 Oct 2017 16:21:33 +0000 (09:21 -0700)] 
experimenting with new Jython

13 months agoadded integration test
Frank Greguska [Fri, 13 Oct 2017 00:58:29 +0000 (17:58 -0700)] 
added integration test

13 months agoremoving out dir
Frank Greguska [Fri, 13 Oct 2017 00:37:47 +0000 (17:37 -0700)] 
removing out dir

13 months agoinitial commit
Frank Greguska [Fri, 13 Oct 2017 00:36:39 +0000 (17:36 -0700)] 
initial commit

13 months agoInitial commit
fgreg [Fri, 13 Oct 2017 00:38:47 +0000 (17:38 -0700)] 
Initial commit