ASF Git Repos - hudi.git/commit

author	Sivabalan Narayanan <sivabala@uber.com>
	Tue, 10 Aug 2021 16:15:37 +0000 (12:15 -0400)
committer	GitHub <noreply@github.com>
	Tue, 10 Aug 2021 16:15:37 +0000 (09:15 -0700)
commit	1196736185272afaeda37328f1c52ccd5c6dc017
tree	72ff45e39ccd243e6a2090b479b377f34d83029e	tree \| snapshot
parent	73d898322b69f8d3831bf209dceb069ea61f5064	commit \| diff

[HUDI-1129] Improving schema evolution support in hudi (#2927)

* Adding support to ingest records with old schema after table's schema is evolved

* Rebasing against latest master

- Trimming test file to be < 800 lines
- Renaming config names

* Addressing feedback

Co-authored-by: Vinoth Chandar <vinoth@apache.org>

22 files changed:

hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionHelper.scala		diff \| blob \| history
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala		diff \| blob \| history
hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java		diff \| blob \| history
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java		diff \| blob \| history
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java		diff \| blob \| history
hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/reader/SparkBasedReader.java		diff \| blob \| history
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/main/java/org/apache/hudi/bootstrap/SparkParquetBootstrapDataProvider.java		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/HoodieSparkSqlWriterSuite.scala		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestHoodieSparkUtils.scala		diff \| blob \| history
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestCOWDataSource.scala		diff \| blob \| history
hudi-utilities/src/main/java/org/apache/hudi/utilities/UtilHelpers.java		diff \| blob \| history
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java		diff \| blob \| history
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java		diff \| blob \| history
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/SourceFormatAdapter.java		diff \| blob \| history
hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/SparkAvroPostProcessor.java		diff \| blob \| history
hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamer.java		diff \| blob \| history
hudi-utilities/src/test/java/org/apache/hudi/utilities/functional/TestHoodieDeltaStreamerBase.java		diff \| blob \| history
hudi-utilities/src/test/java/org/apache/hudi/utilities/testutils/UtilitiesTestBase.java		diff \| blob \| history
hudi-utilities/src/test/resources/delta-streamer-config/source_evolved.avsc	[new file with mode: 0644]	blob

Upserts, Deletes And Incremental Processing on Big Data.

RSS Atom