projects / incubator-hivemall.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: 647c6ae) | patch
[HIVEMALL-267] Drop Spark Dataframe support (SparkSQL remain supported)
authorMakoto Yui <myui@apache.org>
Fri, 4 Oct 2019 05:28:49 +0000 (14:28 +0900)
committerMakoto Yui <myui@apache.org>
Fri, 4 Oct 2019 05:28:49 +0000 (14:28 +0900)
commitff3693d122b6e681b793985f06076a2c56561619
tree60a78a3c1d8d82c01a149bbe880533a32c342eaftree | snapshot
parent647c6ae31ddc0fa290f540716b3026b0e042f39bcommit | diff
[HIVEMALL-267] Drop Spark Dataframe support (SparkSQL remain supported)

## What changes were proposed in this pull request?

Drop Spark Dataframe support (SparkSQL remain supported).

## What type of PR is it?

Hot Fix, Refactoring

## What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-267

## How was this patch tested?

unit tests, manual tests

## Checklist

(Please remove this section if not needed; check `x` for YES, blank for NO)

- [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for your commit?
- [ ] Did you run system tests on Hive (or Spark)?

Author: Makoto Yui <myui@apache.org>

Closes #201 from myui/HIVEMALL-267.
116 files changed:
.gitignore diff | blob | history
bin/format_header.sh diff | blob | history
bin/run_travis_tests.sh diff | blob | history
bin/spark-shell [deleted file] blob | history
conf/spark-defaults.conf [deleted file] blob | history
docs/gitbook/SUMMARY.md diff | blob | history
docs/gitbook/spark/binaryclass/a9a_df.md [deleted file] blob | history
docs/gitbook/spark/getting_started/README.md [deleted file] blob | history
docs/gitbook/spark/getting_started/installation.md diff | blob | history
docs/gitbook/spark/misc/functions.md [deleted file] blob | history
docs/gitbook/spark/misc/misc.md [deleted file] blob | history
docs/gitbook/spark/misc/topk_join.md [deleted file] blob | history
docs/gitbook/spark/regression/e2006_df.md [deleted file] blob | history
pom.xml diff | blob | history
resources/ddl/import-packages.spark [deleted file] blob | history
spark/common/pom.xml [deleted file] blob | history
spark/common/src/main/java/hivemall/dataset/LogisticRegressionDataGeneratorUDTFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/AddBiasUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/AddFeatureIndexUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/ExtractFeatureUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/ExtractWeightUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/SortByFeatureUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/ftvec/scaling/L2NormalizationUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/knn/lsh/MinHashesUDFWrapper.java [deleted file] blob | history
spark/common/src/main/java/hivemall/tools/mapred/RowIdUDFWrapper.java [deleted file] blob | history
spark/common/src/main/scala/hivemall/HivemallException.scala [deleted file] blob | history
spark/common/src/main/scala/org/apache/spark/ml/feature/HivemallLabeledPoint.scala [deleted file] blob | history
spark/pom.xml [deleted file] blob | history
spark/scalastyle-config.xml [deleted file] blob | history
spark/spark-2.2/bin/mvn-zinc [deleted file] blob | history
spark/spark-2.2/extra-src/README.md [deleted file] blob | history
spark/spark-2.2/extra-src/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala [deleted file] blob | history
spark/spark-2.2/pom.xml [deleted file] blob | history
spark/spark-2.2/src/main/java/hivemall/xgboost/XGBoostOptions.scala [deleted file] blob | history
spark/spark-2.2/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister [deleted file] blob | history
spark/spark-2.2/src/main/resources/log4j.properties [deleted file] blob | history
spark/spark-2.2/src/main/scala/hivemall/tools/RegressionDatagen.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/catalyst/expressions/EachTopK.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/JoinTopK.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/catalyst/utils/InternalRowPriorityQueue.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/execution/UserProvidedPlanner.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/execution/datasources/csv/csvExpressions.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinTopKExec.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/hive/HivemallGroupedDataset.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/hive/HivemallUtils.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/hive/internal/HivemallOpsImpl.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/sql/hive/source/XGBoostFileFormat.scala [deleted file] blob | history
spark/spark-2.2/src/main/scala/org/apache/spark/streaming/HivemallStreamingOps.scala [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/README.md [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/complex.seq [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/episodes.avro [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/json.txt [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/kv1.txt [deleted file] blob | history
spark/spark-2.2/src/test/resources/data/files/kv3.txt [deleted file] blob | history
spark/spark-2.2/src/test/resources/log4j.properties [deleted file] blob | history
spark/spark-2.2/src/test/scala/hivemall/mix/server/MixServerSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/hivemall/tools/RegressionDatagenSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/SparkFunSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/ml/feature/HivemallLabeledPointSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/QueryTest.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/execution/benchmark/BenchmarkBase.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/HiveUdfSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/HivemallOpsSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/ModelMixingSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/XGBoostSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/benchmark/MiscBenchmark.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/test/HivemallFeatureQueryTest.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/hive/test/TestHiveSingleton.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/test/SQLTestData.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/sql/test/VectorQueryTest.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/streaming/HivemallOpsWithFeatureSuite.scala [deleted file] blob | history
spark/spark-2.2/src/test/scala/org/apache/spark/test/TestUtils.scala [deleted file] blob | history
spark/spark-2.3/bin/mvn-zinc [deleted file] blob | history
spark/spark-2.3/extra-src/README.md [deleted file] blob | history
spark/spark-2.3/extra-src/hive/src/main/scala/org/apache/spark/sql/hive/HiveShim.scala [deleted file] blob | history
spark/spark-2.3/pom.xml [deleted file] blob | history
spark/spark-2.3/src/main/java/hivemall/xgboost/XGBoostOptions.scala [deleted file] blob | history
spark/spark-2.3/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister [deleted file] blob | history
spark/spark-2.3/src/main/resources/log4j.properties [deleted file] blob | history
spark/spark-2.3/src/main/scala/hivemall/tools/RegressionDatagen.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/catalyst/expressions/EachTopK.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/JoinTopK.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/catalyst/utils/InternalRowPriorityQueue.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/execution/UserProvidedPlanner.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/execution/datasources/csv/csvExpressions.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinTopKExec.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/HivemallGroupedDataset.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/HivemallOps.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/HivemallUtils.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/internal/HivemallOpsImpl.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/sql/hive/source/XGBoostFileFormat.scala [deleted file] blob | history
spark/spark-2.3/src/main/scala/org/apache/spark/streaming/HivemallStreamingOps.scala [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/README.md [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/complex.seq [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/episodes.avro [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/json.txt [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/kv1.txt [deleted file] blob | history
spark/spark-2.3/src/test/resources/data/files/kv3.txt [deleted file] blob | history
spark/spark-2.3/src/test/resources/log4j.properties [deleted file] blob | history
spark/spark-2.3/src/test/scala/hivemall/mix/server/MixServerSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/hivemall/tools/RegressionDatagenSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/ml/feature/HivemallLabeledPointSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/execution/benchmark/BenchmarkBaseAccessor.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/HiveUdfSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/HivemallOpsSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/ModelMixingSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/XGBoostSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/benchmark/MiscBenchmark.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/hive/test/HivemallFeatureQueryTest.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/sql/test/VectorQueryTest.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/streaming/HivemallOpsWithFeatureSuite.scala [deleted file] blob | history
spark/spark-2.3/src/test/scala/org/apache/spark/test/TestUtils.scala [deleted file] blob | history
src/site/markdown/overview.md diff | blob | history
Apache Hivemall (incubating)