crunch.git
5 months agoCRUNCH-671: Failed to generate reports using "mvn site" master
Jun He [Thu, 9 Aug 2018 05:49:09 +0000 (05:49 +0000)] 
CRUNCH-671: Failed to generate reports using "mvn site"

Crunch build failed due to "ClassNotFound" in doxia.
This is caused by maven-project-info-reports-plugin updated to 3.0.0, depends on
doxia-site-renderer 1.8 (which has org.apache.maven.doxia.siterenderer.DocumentContent
this class), while maven-site-plugin:3.3 depends on doxia-site-renderer:1.4 (which
doesn't have org.apache.maven.doxia.siterenderer.DocumentContent)
Specify maven-site-plugin to 3.7 can resolve this.

Signed-off-by: Jun He <jun.he@linaro.org>
Signed-off-by: Josh Wills <jwills@apache.org>
5 months agoCRUNCH-619: Update to HBase 2.0.1. Contributed by Attila Sasvari.
Josh Wills [Mon, 23 Jul 2018 20:31:00 +0000 (13:31 -0700)] 
CRUNCH-619: Update to HBase 2.0.1. Contributed by Attila Sasvari.

8 months agoCRUNCH-669: Add an option to disable temp dir deletion in the finalize() method of...
Josh Wills [Mon, 30 Apr 2018 18:47:15 +0000 (11:47 -0700)] 
CRUNCH-669: Add an option to disable temp dir deletion in the finalize() method of a DistributedPipeline

9 months agoCRUNCH-668: Support globbing patterns in From#avroFile
Clément MATHIEU [Tue, 27 Mar 2018 15:55:15 +0000 (17:55 +0200)] 
CRUNCH-668: Support globbing patterns in From#avroFile

Signed-off-by: Josh Wills <jwills@apache.org>
10 months agoFix HCatSourceITSpec.testBasic
Clément MATHIEU [Tue, 6 Mar 2018 16:47:48 +0000 (17:47 +0100)] 
Fix HCatSourceITSpec.testBasic

Signed-off-by: Josh Wills <jwills@apache.org>
10 months agoCRUNCH-665: Add crunch.max.poll.interval property
Clément MATHIEU [Wed, 7 Mar 2018 09:13:51 +0000 (10:13 +0100)] 
CRUNCH-665: Add crunch.max.poll.interval property

Signed-off-by: Josh Wills <jwills@apache.org>
11 months agoCRUNCH-664 Fixes HBase configuration properties being overwritten
Nathan Schile [Mon, 5 Feb 2018 15:08:46 +0000 (09:08 -0600)] 
CRUNCH-664 Fixes HBase configuration properties being overwritten

Signed-off-by: Josh Wills <jwills@apache.org>
11 months agoExpose combine file split file path via Hadoop config
Ben Roling [Wed, 24 Jan 2018 16:40:18 +0000 (10:40 -0600)] 
Expose combine file split file path via Hadoop config

Signed-off-by: Josh Wills <jwills@apache.org>
11 months agoCRUNCH-662: Updated KafkaRecordReader to better handle errors, empty reads and approp...
Bryan Baugher [Wed, 24 Jan 2018 20:14:31 +0000 (14:14 -0600)] 
CRUNCH-662: Updated KafkaRecordReader to better handle errors, empty reads and appropriately retry

Signed-off-by: Josh Wills <jwills@apache.org>
12 months agoCRUNCH-661: Make DataBaseSource.Builder methods public
Josh Wills [Thu, 18 Jan 2018 21:11:26 +0000 (13:11 -0800)] 
CRUNCH-661: Make DataBaseSource.Builder methods public

13 months agoCRUNCH-654: KafkaSource should use the new Kafka Consumer API instead of the SimpleCo...
Josh Wills [Mon, 11 Dec 2017 17:56:38 +0000 (09:56 -0800)] 
CRUNCH-654: KafkaSource should use the new Kafka Consumer API instead of the SimpleConsumer. Contributed by Bryan Baugher.

13 months agoCRUNCH-340: added HCatSource & HCatTarget
Stephen Durfey [Mon, 4 Dec 2017 16:49:59 +0000 (10:49 -0600)] 
CRUNCH-340: added HCatSource & HCatTarget

Signed-off-by: Josh Wills <jwills@apache.org>
13 months agoCRUNCH-659: updated hive dependency to 2.1
Stephen Durfey [Thu, 7 Dec 2017 15:55:56 +0000 (09:55 -0600)] 
CRUNCH-659: updated hive dependency to 2.1

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
14 months agoCRUNCH-652: Fix to make the SourceTargetHelperTest less flakey on hadoop 3.0.0. Contr...
Josh Wills [Fri, 27 Oct 2017 04:09:27 +0000 (21:09 -0700)] 
CRUNCH-652: Fix to make the SourceTargetHelperTest less flakey on hadoop 3.0.0. Contributed by Gergo Repas.

14 months agoCRUNCH-653: Created KafkaSource that provides ConsumerRecord messages
Bryan Baugher [Wed, 16 Aug 2017 21:19:42 +0000 (16:19 -0500)] 
CRUNCH-653: Created KafkaSource that provides ConsumerRecord messages

Signed-off-by: Josh Wills <jwills@apache.org>
20 months agoCRUNCH-647: Remove obsolete jackson dependencies
Josh Wills [Fri, 12 May 2017 16:52:49 +0000 (09:52 -0700)] 
CRUNCH-647: Remove obsolete jackson dependencies

20 months agoCRUNCH-644 Supply preferred node for HFile writes
Gabriel Reid [Thu, 27 Apr 2017 12:52:16 +0000 (14:52 +0200)] 
CRUNCH-644 Supply preferred node for HFile writes

Designate the preferred HDFS data node when creating HFiles for
bulk load to improve data locality of the created HFiles.

21 months agoCRUNCH-618: Run on Spark 2. Contributed by Gergő Pásztor.
Tom White [Thu, 13 Apr 2017 15:10:23 +0000 (16:10 +0100)] 
CRUNCH-618: Run on Spark 2. Contributed by Gergő Pásztor.

21 months agoCRUNCH-642 Enable GroupingOptions for Distinct operations.
Xavier Talpe [Thu, 13 Apr 2017 05:52:43 +0000 (07:52 +0200)] 
CRUNCH-642 Enable GroupingOptions for Distinct operations.

This fixes the existing call for numReducers as it was not working as
intended for non-memory PCollections due to using an invalid amount
of numReducers. To increase flexibility when using the API,
another call was added that allow to directly pass the GroupingOptions.

Signed-off-by: Josh Wills <jwills@apache.org>
21 months agoCRUNCH-641: Wrong decimal format in dot files. Contributed by Gergő Pásztor.
Tom White [Wed, 12 Apr 2017 14:03:41 +0000 (15:03 +0100)] 
CRUNCH-641: Wrong decimal format in dot files. Contributed by Gergő Pásztor.

21 months agoCRUNCH-642 Enable numReducers option for Distinct operations.
Xavier Talpe [Mon, 10 Apr 2017 13:51:32 +0000 (15:51 +0200)] 
CRUNCH-642 Enable numReducers option for Distinct operations.

Signed-off-by: Josh Wills <jwills@apache.org>
21 months agoCRUNCH-636: amend Make replication factor for temporary files configurable
Attila Sasvari [Thu, 23 Mar 2017 20:35:36 +0000 (21:35 +0100)] 
CRUNCH-636: amend Make replication factor for temporary files configurable

Signed-off-by: Josh Wills <jwills@apache.org>
22 months agoCRUNCH-636: Make replication factor for temporary files configurable
Attila Sasvari [Mon, 20 Mar 2017 10:17:55 +0000 (11:17 +0100)] 
CRUNCH-636: Make replication factor for temporary files configurable

Signed-off-by: Josh Wills <jwills@apache.org>
22 months agoCRUNCH-638: Improve dot file generation for better supportability. Contributed by...
Tom White [Tue, 7 Mar 2017 14:38:52 +0000 (14:38 +0000)] 
CRUNCH-638: Improve dot file generation for better supportability. Contributed by Gergő Pásztor.

23 months agoCRUNCH-633: Remove the commons-httpclient:commons-httpclient dependency. Contributed...
Tom White [Mon, 20 Feb 2017 10:28:05 +0000 (10:28 +0000)] 
CRUNCH-633: Remove the commons-httpclient:commons-httpclient dependency. Contributed by Gergő Pásztor.

23 months agoCRUNCH-634 Fix typo in log message
Gabriel Reid [Mon, 13 Feb 2017 18:57:32 +0000 (19:57 +0100)] 
CRUNCH-634 Fix typo in log message

Contributed by Attila Sasvari

23 months agoCRUNCH-628: Upgraded to Kafka 0.10.0.x
Micah Whitacre [Wed, 7 Dec 2016 02:50:02 +0000 (21:50 -0500)] 
CRUNCH-628: Upgraded to Kafka 0.10.0.x

23 months ago[maven-release-plugin] prepare for next development iteration
Josh Wills [Sun, 5 Feb 2017 19:22:06 +0000 (11:22 -0800)] 
[maven-release-plugin] prepare for next development iteration

23 months ago[maven-release-plugin] prepare branch apache-crunch-0.15
Josh Wills [Sun, 5 Feb 2017 19:22:05 +0000 (11:22 -0800)] 
[maven-release-plugin] prepare branch apache-crunch-0.15

23 months agoCRUNCH-632: Added support for compressed CSVSource files.
Micah Whitacre [Thu, 12 Jan 2017 02:51:26 +0000 (20:51 -0600)] 
CRUNCH-632: Added support for compressed CSVSource files.

CRUNCH-632: Wrote simple test showing it now working on compressed CSV file.

Signed-off-by: Josh Wills <jwills@apache.org>
2 years agoMerge branch 'CRUNCH-630'
Micah Whitacre [Thu, 12 Jan 2017 01:53:05 +0000 (19:53 -0600)] 
Merge branch 'CRUNCH-630'

2 years agoCRUNCH-629: Kafka source pulling is aggressive
Brian Tieman [Tue, 13 Dec 2016 15:01:08 +0000 (09:01 -0600)] 
CRUNCH-629: Kafka source pulling is aggressive

Added some parenthesis to force proper order of operations in KafkaRecordReader.

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-630: set a better default for the situation where offsets are out of range.
Micah Whitacre [Tue, 3 Jan 2017 17:39:31 +0000 (11:39 -0600)] 
CRUNCH-630: set a better default for the situation where offsets are out of range.

2 years agoQuick and Dirty Workaround for Crunch DistCache
Dimitry Goldin [Fri, 14 Oct 2016 16:39:41 +0000 (18:39 +0200)] 
Quick and Dirty Workaround for Crunch DistCache

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
Signed-off-by: Josh Wills <jwills@apache.org>
2 years agoCRUNCH-622: From.avroFile fails if path not on default filesystem. Contributed by...
Josh Wills [Sat, 3 Dec 2016 19:56:59 +0000 (11:56 -0800)] 
CRUNCH-622: From.avroFile fails if path not on default filesystem. Contributed by Micah Whitacre.

2 years agoCRUNCH-620: Reduce "isn't a known config" warnings by slimming down ConsumerConfig...
Stefan Mendoza [Tue, 13 Sep 2016 03:38:41 +0000 (22:38 -0500)] 
CRUNCH-620: Reduce "isn't a known config" warnings by slimming down ConsumerConfig properties

Resolved by tagging the Kafka connection properties so that the Kafka Consumers can be built with slimmer ConsumerConfig properties.

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-625: Add missing .union implementations for LTables with LTables and PTables
David Whiting [Thu, 20 Oct 2016 14:17:00 +0000 (16:17 +0200)] 
CRUNCH-625: Add missing .union implementations for LTables with LTables and PTables

2 years agoCRUNCH-621: Added check into hasPendingData to check if there is a large number of...
Micah Whitacre [Tue, 13 Sep 2016 15:35:35 +0000 (10:35 -0500)] 
CRUNCH-621: Added check into hasPendingData to check if there is a large number of requests with no data to make sure there is still data there.

2 years agoCRUNCH-623: Improves Javadoc of PTable#cogroup
Nathan Schile [Thu, 29 Sep 2016 21:24:14 +0000 (16:24 -0500)] 
CRUNCH-623: Improves Javadoc of PTable#cogroup

Signed-off-by: Josh Wills <jwills@apache.org>
2 years agoCRUNCH-617: Support defensively handling null when partition leader cannot be found.
Micah Whitacre [Tue, 6 Sep 2016 20:55:56 +0000 (15:55 -0500)] 
CRUNCH-617: Support defensively handling null when partition leader cannot be found.

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-616: Replace (possibly copyrighted) Maugham text with Dickens. Contributed...
Tom White [Thu, 8 Sep 2016 13:12:30 +0000 (14:12 +0100)] 
CRUNCH-616: Replace (possibly copyrighted) Maugham text with Dickens. Contributed by Sean Owen.

Remove non-applicable Project Gutenberg license. Adjust lots of tests to match new text.

2 years agoCRUNCH-601: Handle empty PCollections correctly in Crunch-on-Spark. Created by Micah...
Josh Wills [Wed, 24 Aug 2016 17:59:14 +0000 (10:59 -0700)] 
CRUNCH-601: Handle empty PCollections correctly in Crunch-on-Spark. Created by Micah Whitacre,
Mikael Goldmann, and Josh Wills.

2 years agoCRUNCH-519: Add more detail to plan dot file. Contributed by Ron Hashimshony.
Josh Wills [Wed, 24 Aug 2016 03:07:23 +0000 (20:07 -0700)] 
CRUNCH-519: Add more detail to plan dot file. Contributed by Ron Hashimshony.

2 years agoCRUNCH-604: Avoid expensive Writables.reloadWritableComparableCodes
Micah Whitacre [Tue, 2 Aug 2016 21:29:55 +0000 (16:29 -0500)] 
CRUNCH-604: Avoid expensive Writables.reloadWritableComparableCodes

2 years agoCRUNCH-611: Corrected files that were missing the APL headers.
Micah Whitacre [Tue, 2 Aug 2016 20:58:29 +0000 (15:58 -0500)] 
CRUNCH-611: Corrected files that were missing the APL headers.

2 years agoCRUNCH-611: Added API for Offset reading/writing along with a simple implementation...
Micah Whitacre [Wed, 13 Jul 2016 15:18:17 +0000 (10:18 -0500)] 
CRUNCH-611: Added API for Offset reading/writing along with a simple implementation that supports doing it from hdfs.

Signed-off-by: Micah Whitacre <mkwhit@apache.org>
2 years agoCRUNCH-614: Fix HFileUtils.writeToHFilesForIncrementalLoad slowed dramatically by...
Josh Wills [Sat, 30 Jul 2016 00:47:09 +0000 (17:47 -0700)] 
CRUNCH-614: Fix HFileUtils.writeToHFilesForIncrementalLoad slowed dramatically by copying KeyValue byte array. Contributed by Ben Roling.

2 years agoCRUNCH-613: Fix FileTargetImplTest.testHandleOutputsMovesFilesToDestination instability
Clément MATHIEU [Tue, 19 Jul 2016 19:30:35 +0000 (21:30 +0200)] 
CRUNCH-613: Fix FileTargetImplTest.testHandleOutputsMovesFilesToDestination instability

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
CRUNCH-613: Fixed up the test to consolidate constants used.

2 years agoCRUNCH-612: Add support of private ctors to AvroDeepCopier
Clément MATHIEU [Tue, 19 Jul 2016 19:01:20 +0000 (21:01 +0200)] 
CRUNCH-612: Add support of private ctors to AvroDeepCopier

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-609: Improved KafkaRecordReader to keep retrying when the range of offsets...
Micah Whitacre [Tue, 28 Jun 2016 20:44:15 +0000 (15:44 -0500)] 
CRUNCH-609: Improved KafkaRecordReader to keep retrying when the range of offsets has not been fully consumed.

2 years agoCRUNCH-606: Handle setting version correctly and removed stray System.out in test.
Micah Whitacre [Mon, 23 May 2016 20:13:02 +0000 (15:13 -0500)] 
CRUNCH-606: Handle setting version correctly and removed stray System.out in test.

2 years agoCRUNCH-606: Kafka Source for Crunch which supports reading data as BytesWritable
Micah Whitacre [Mon, 11 Apr 2016 14:47:33 +0000 (09:47 -0500)] 
CRUNCH-606: Kafka Source for Crunch which supports reading data as BytesWritable

* Some of the code contributed by Bryan Baugher and Andrew Olson

2 years agoCRUNCH-608 Write Bloom filters in HFiles
Gabriel Reid [Tue, 10 May 2016 09:02:11 +0000 (11:02 +0200)] 
CRUNCH-608 Write Bloom filters in HFiles

Use a correctly-configured StoreFile.Writer (instead of HFile.Writer)
for writing HFiles so that Bloom filter data is also included in
the written HFiles.

2 years agoCRUNCH-607 Allow collection reuse in MemPipeline
Gabriel Reid [Mon, 2 May 2016 15:31:20 +0000 (17:31 +0200)] 
CRUNCH-607 Allow collection reuse in MemPipeline

Prevent SingleUseIterable from throwing an IllegalArgumentException
when legal reuse of PGroupedCollections are done with the
MemPipeline.

This simply prevents materializing the transformed contents of
a MemCollection until it is iterated over.

2 years ago[maven-release-plugin] prepare for next development iteration
Josh Wills [Sun, 24 Apr 2016 02:23:02 +0000 (19:23 -0700)] 
[maven-release-plugin] prepare for next development iteration

2 years ago[maven-release-plugin] prepare branch apache-crunch-0.14
Josh Wills [Sun, 24 Apr 2016 02:23:01 +0000 (19:23 -0700)] 
[maven-release-plugin] prepare branch apache-crunch-0.14

2 years agoCRUNCH-579: Supported access to counters from original TaskContext
mkwhitacre [Mon, 23 Nov 2015 00:07:30 +0000 (18:07 -0600)] 
CRUNCH-579: Supported access to counters from original TaskContext

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-600: pass job credentials when building multiple outputs
Igor Bernstein [Sun, 10 Apr 2016 19:42:10 +0000 (15:42 -0400)] 
CRUNCH-600: pass job credentials when building multiple outputs

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-599: Fix increment and incrementIf methods in crunch-lambda so they also emit...
David Whiting [Thu, 31 Mar 2016 10:06:45 +0000 (12:06 +0200)] 
CRUNCH-599: Fix increment and incrementIf methods in crunch-lambda so they also emit the incoming element

2 years agoCRUNCH-597: Upgrade to Parquet 1.8.1
Josh Wills [Thu, 24 Mar 2016 16:55:16 +0000 (09:55 -0700)] 
CRUNCH-597: Upgrade to Parquet 1.8.1

2 years agoCRUNCH-596 Support right-outer bloom join
tworec [Fri, 4 Mar 2016 17:58:01 +0000 (18:58 +0100)] 
CRUNCH-596 Support right-outer bloom join

Signed-off-by: Gabriel Reid <greid@apache.org>
2 years agoAdded the ability to specify the amount of reducers when doing a sharded join.
Joel [Thu, 24 Dec 2015 16:55:03 +0000 (17:55 +0100)] 
Added the ability to specify the amount of reducers when doing a sharded join.

Signed-off-by: Josh Wills <jwills@apache.org>
2 years agoCRUNCH-586: Make SparkPipeline support reads from HBaseSourceTargets
Josh Wills [Tue, 19 Jan 2016 06:33:23 +0000 (22:33 -0800)] 
CRUNCH-586: Make SparkPipeline support reads from HBaseSourceTargets

2 years agochanges to support only affected regions during hfile write
Stephen Durfey [Fri, 12 Feb 2016 23:14:01 +0000 (17:14 -0600)] 
changes to support only affected regions during hfile write

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
2 years agoCRUNCH-594: Moved the crunch-lambda to a java8 activated profile.
Micah Whitacre [Fri, 12 Feb 2016 03:54:59 +0000 (21:54 -0600)] 
CRUNCH-594: Moved the crunch-lambda to a java8 activated profile.

2 years agoCRUNCH-593: Added exclusions for iml and target directory.
Micah Whitacre [Fri, 12 Feb 2016 03:38:51 +0000 (21:38 -0600)] 
CRUNCH-593: Added exclusions for iml and target directory.

2 years agoCRUNCH-592: Job fails for null ByteBuffer value in Avro tables.
Tom White [Wed, 10 Feb 2016 16:02:31 +0000 (16:02 +0000)] 
CRUNCH-592: Job fails for null ByteBuffer value in Avro tables.

2 years agoCRUNCH-590: Fix CSVInputFormat to work with S3
Josh Wills [Tue, 2 Feb 2016 19:05:04 +0000 (11:05 -0800)] 
CRUNCH-590: Fix CSVInputFormat to work with S3

2 years agoCRUNCH-582: Upgrade Crunch Guava to 14.0.1
Josh Wills [Mon, 7 Dec 2015 02:13:00 +0000 (18:13 -0800)] 
CRUNCH-582: Upgrade Crunch Guava to 14.0.1

3 years agoMake replication factor for DistCache configurable
Steffen Grohsschmiedt [Thu, 21 Jan 2016 09:42:07 +0000 (10:42 +0100)] 
Make replication factor for DistCache configurable

Add configuration option to set the replication factor for files
distributed with the DistCache helper class.

Signed-off-by: mkwhitacre <mkwhitacre@gmail.com>
3 years agoCRUNCH-587: Add missing filter(), filterByKey() and filterByValue() functions from...
David Whiting [Mon, 18 Jan 2016 14:07:13 +0000 (15:07 +0100)] 
CRUNCH-587: Add missing filter(), filterByKey() and filterByValue() functions from Lambda LTable implementation

3 years agoJava 8 lambda support for Apache Crunch.
David Whiting [Sat, 2 Jan 2016 10:28:34 +0000 (11:28 +0100)] 
Java 8 lambda support for Apache Crunch.

Remove lambda support from crunch-core, and instead implement a new module called crunch-lambda.
This will allow full use of Java 8 features in implementing support for lambda expressions and
method references, without requiring a dependency on Java 8 for crunch-core. Pthings are wrapped
into analagous Lthings which can be operated on with an API inspired both by the existing Crunch
API and the Java 8 streams API.

3 years agoCRUNCH-584: Add missing mapValues in PGroupedTable for Java 8 lambdas
David Whiting [Thu, 10 Dec 2015 16:20:35 +0000 (17:20 +0100)] 
CRUNCH-584: Add missing mapValues in PGroupedTable for Java 8 lambdas

3 years agoCRUNCH-583: Scrunch classloader failure in distcache
Tom White [Thu, 10 Dec 2015 13:19:24 +0000 (13:19 +0000)] 
CRUNCH-583: Scrunch classloader failure in distcache

3 years agoCRUNCH-580: Use thread pools in org.apache.crunch.io.impl.FileTargetImpl#handleOutput...
Jeff Quinn [Wed, 25 Nov 2015 18:11:04 +0000 (10:11 -0800)] 
CRUNCH-580: Use thread pools in org.apache.crunch.io.impl.FileTargetImpl#handleOutputs for file renaming.

Signed-off-by: Josh Wills <jwills@apache.org>
3 years agoCRUNCH-578: Add support for mutable collection type serialization to Scrunch
Josh Wills [Tue, 17 Nov 2015 01:35:48 +0000 (17:35 -0800)] 
CRUNCH-578: Add support for mutable collection type serialization to Scrunch

3 years agoCRUNCH-576 Add multi-scan methods to FromHBase
Gabriel Reid [Fri, 6 Nov 2015 13:25:21 +0000 (14:25 +0100)] 
CRUNCH-576 Add multi-scan methods to FromHBase

Add factory methods which allow reading a PCollection from HBase
that uses multiple Scans.

3 years agoCRUNCH-577 Use getLongBytes() to correctly parse dfs block size.
Tomáš Čechal [Tue, 10 Nov 2015 11:11:13 +0000 (12:11 +0100)] 
CRUNCH-577 Use getLongBytes() to correctly parse dfs block size.

Signed-off-by: Josh Wills <jwills@apache.org>
3 years agoCRUNCH-561: Scrunch case classes fail in the REPL.
Tom White [Tue, 27 Oct 2015 09:22:07 +0000 (09:22 +0000)] 
CRUNCH-561: Scrunch case classes fail in the REPL.

3 years agoCRUNCH-515: Decrease collision probability on temp dir cleanup. Contributed by Sean...
Josh Wills [Mon, 19 Oct 2015 19:15:10 +0000 (12:15 -0700)] 
CRUNCH-515: Decrease collision probability on temp dir cleanup. Contributed by Sean Owen.

3 years agoCRUNCH-574: Upgraded the commons-lang dependency to 2.6 courtesy of Sean Owen (srowen)
Micah Whitacre [Mon, 19 Oct 2015 13:57:47 +0000 (08:57 -0500)] 
CRUNCH-574: Upgraded the commons-lang dependency to 2.6 courtesy of Sean Owen (srowen)

3 years agoCRUNCH-571: Scrunch functions fail serialization check in the REPL
Tom White [Mon, 19 Oct 2015 09:21:36 +0000 (10:21 +0100)] 
CRUNCH-571: Scrunch functions fail serialization check in the REPL

3 years agoCRUNCH-562: Support one output file per key for Parquet.
Tom White [Mon, 19 Oct 2015 09:05:27 +0000 (10:05 +0100)] 
CRUNCH-562: Support one output file per key for Parquet.

3 years agoCRUNCH-568: Don't use a null key in the Aggregators.aggregate implementation
Josh Wills [Tue, 6 Oct 2015 21:30:24 +0000 (14:30 -0700)] 
CRUNCH-568: Don't use a null key in the Aggregators.aggregate implementation

3 years agoCRUNCH-565_CSVInputFormat-Configuration-Defensiveness
‘Mac [Fri, 2 Oct 2015 00:30:24 +0000 (19:30 -0500)] 
CRUNCH-565_CSVInputFormat-Configuration-Defensiveness

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
3 years agoCRUNCH-567 Fix potential NPE on close
Gabriel Reid [Sat, 3 Oct 2015 15:00:15 +0000 (17:00 +0200)] 
CRUNCH-567 Fix potential NPE on close

Prevent a potential NPE on calling close on an incompletely
initialized AvroRecordReader or HFileRecordReader.

Contributed by Sean Owen.

3 years agoCRUNCH-563: Add support for BigDecimal aggregators. Contributed by Vasu Doppalapudi.
Josh Wills [Sun, 27 Sep 2015 18:45:03 +0000 (11:45 -0700)] 
CRUNCH-563: Add support for BigDecimal aggregators. Contributed by Vasu Doppalapudi.

3 years agoCRUNCH-559: Enabling Spark integration tests
Josh Wills [Wed, 9 Sep 2015 05:09:21 +0000 (22:09 -0700)] 
CRUNCH-559: Enabling Spark integration tests

Signed-off-by: Micah Whitacre <mkwhit@gmail.com>
3 years agoCRUNCH-558: Set the Pipeline name as accumulator name
Micah Whitacre [Thu, 3 Sep 2015 12:47:37 +0000 (07:47 -0500)] 
CRUNCH-558: Set the Pipeline name as accumulator name

3 years agoCRUNCH-538: Java lambdas for Crunch business logic.
Josh Wills [Wed, 1 Jul 2015 22:20:50 +0000 (15:20 -0700)] 
CRUNCH-538: Java lambdas for Crunch business logic.

3 years agoCRUNCH-556: Fix partitioner configuration in Crunch-on-Spark
Josh Wills [Thu, 13 Aug 2015 05:15:58 +0000 (22:15 -0700)] 
CRUNCH-556: Fix partitioner configuration in Crunch-on-Spark

3 years agoCRUNCH-554 Add MAX_COMPARABLES and MIN_COMPARABLES
Gabriel Reid [Wed, 29 Jul 2015 08:26:52 +0000 (10:26 +0200)] 
CRUNCH-554 Add MAX_COMPARABLES and MIN_COMPARABLES

3 years ago[maven-release-plugin] prepare for next development iteration
Josh Wills [Thu, 30 Jul 2015 19:06:24 +0000 (14:06 -0500)] 
[maven-release-plugin] prepare for next development iteration

3 years ago[maven-release-plugin] prepare branch apache-crunch-0.13
Josh Wills [Thu, 30 Jul 2015 19:06:24 +0000 (14:06 -0500)] 
[maven-release-plugin] prepare branch apache-crunch-0.13

3 years agoCRUNCH-550: Removed deprecations in crunch-hbase also added
Micah Whitacre [Tue, 28 Jul 2015 01:57:07 +0000 (20:57 -0500)] 
CRUNCH-550: Removed deprecations in crunch-hbase also added
 support for TableName.

3 years agoCRUNCH-553: Fix record drop issue that can occur w/From.formattedFile TableSources
Josh Wills [Tue, 28 Jul 2015 01:45:23 +0000 (18:45 -0700)] 
CRUNCH-553: Fix record drop issue that can occur w/From.formattedFile TableSources

3 years agoCRUNCH-551: Make the use of Configuration objects consistent in CrunchInputSplit...
Josh Wills [Mon, 27 Jul 2015 23:00:49 +0000 (16:00 -0700)] 
CRUNCH-551: Make the use of Configuration objects consistent in CrunchInputSplit and CrunchRecordReader

3 years agoCRUNCH-552: Add support/tests for Parquet files w/Crunch on Spark
Josh Wills [Tue, 28 Jul 2015 00:13:30 +0000 (17:13 -0700)] 
CRUNCH-552: Add support/tests for Parquet files w/Crunch on Spark

3 years agoCRUNCH-547: Properly handle nullability for Avro union types
Josh Wills [Wed, 22 Jul 2015 17:38:56 +0000 (10:38 -0700)] 
CRUNCH-547: Properly handle nullability for Avro union types

3 years agoCRUNCH-548: Have the AvroReflectDeepCopier use the class of the source object when...
Josh Wills [Wed, 22 Jul 2015 17:45:19 +0000 (10:45 -0700)] 
CRUNCH-548: Have the AvroReflectDeepCopier use the class of the source object when constructing new instances instead of the target class (which might be an interface/abstract class)