[HIVEMALL-276] Stable support for XGBoost v0.90
authorMakoto Yui <myui@apache.org>
Fri, 22 Nov 2019 15:56:36 +0000 (00:56 +0900)
committerMakoto Yui <myui@apache.org>
Fri, 22 Nov 2019 15:56:36 +0000 (00:56 +0900)
commit54e1d7da67690b17640809e60c7b251fdc82308a
treeae2be1a7cca46fcff4311816474e3b5054c9a45f
parent86409b4347582e3de1c02fc99a20e8d7bb7706cf
[HIVEMALL-276] Stable support for XGBoost v0.90

## What changes were proposed in this pull request?

- Fix xgboost module to create DMatrix from CSRMatrix
- Support xgboost v0.90 hyperparameters
- Replace xgboost4j with [xgboost-predictor](https://github.com/komiya-atsushi/xgboost-predictor-java) for prediction
- Add documentation about Xgboost

## What type of PR is it?

Refactoring, Improvement

## What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-276
https://issues.apache.org/jira/browse/HIVEMALL-275
https://issues.apache.org/jira/browse/HIVEMALL-279
https://issues.apache.org/jira/browse/HIVEMALL-272
https://issues.apache.org/jira/browse/HIVEMALL-27

## How to use this feature?

as described in [user guide](http://hivemall.apache.org/userguide/index.html).

## How was this patch tested?

unit tests and manual tests on EMR

## Checklist

- [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for your commit?
- [x] Did you run system tests on Hive (or Spark)?

Author: Makoto Yui <myui@apache.org>

Closes #213 from myui/HIVEMALL-275-2.
48 files changed:
core/src/main/java/hivemall/UDTFWithOptions.java
core/src/main/java/hivemall/classifier/GeneralClassifierUDTF.java
core/src/main/java/hivemall/tools/array/ArrayAvgGenericUDAF.java
core/src/main/java/hivemall/utils/hadoop/HiveUtils.java
core/src/main/java/hivemall/utils/hadoop/WritableUtils.java
core/src/main/java/hivemall/utils/io/IOUtils.java
core/src/main/java/hivemall/utils/lang/ArrayUtils.java
core/src/main/java/hivemall/utils/lang/ObjectUtils.java
core/src/main/java/hivemall/utils/lang/OptionUtils.java [new file with mode: 0644]
core/src/test/java/hivemall/utils/io/IOUtilsTest.java [new file with mode: 0644]
dist/pom.xml
docs/gitbook/SUMMARY.md
docs/gitbook/binaryclass/news20b_xgboost.md [new file with mode: 0644]
docs/gitbook/misc/funcs.md
docs/gitbook/multiclass/iris_xgboost.md [new file with mode: 0644]
docs/gitbook/multiclass/news20_dataset.md
docs/gitbook/multiclass/news20_pa.md
docs/gitbook/multiclass/news20_scw.md
docs/gitbook/multiclass/news20_xgboost.md [new file with mode: 0644]
docs/gitbook/regression/e2006_xgboost.md [new file with mode: 0644]
pom.xml
resources/ddl/define-additional.hive
resources/ddl/define-all-as-permanent.hive
tools/hivemall-docs/src/main/java/hivemall/docs/FuncsListGeneratorMojo.java
xgboost/pom.xml
xgboost/src/main/java/hivemall/xgboost/XGBoostBaseUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/XGBoostBatchPredictUDTF.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/XGBoostOnlinePredictUDTF.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/XGBoostPredictOneUDTF.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/XGBoostPredictTripleUDTF.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/XGBoostPredictUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/XGBoostTrainUDTF.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/XGBoostUtils.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/XGBoostVersionUDF.java
xgboost/src/main/java/hivemall/xgboost/classification/XGBoostBinaryClassifierUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/classification/XGBoostMulticlassClassifierUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/regression/XGBoostRegressionUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/tools/XGBoostMulticlassPredictUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/tools/XGBoostPredictUDTF.java [deleted file]
xgboost/src/main/java/hivemall/xgboost/utils/DMatrixBuilder.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/utils/DenseDMatrixBuilder.java [new file with mode: 0644]
xgboost/src/main/java/hivemall/xgboost/utils/NativeLibLoader.java [moved from xgboost/src/main/java/hivemall/xgboost/NativeLibLoader.java with 99% similarity]
xgboost/src/main/java/hivemall/xgboost/utils/SparseDMatrixBuilder.java
xgboost/src/main/java/hivemall/xgboost/utils/XGBoostUtils.java [new file with mode: 0644]
xgboost/src/test/java/hivemall/TestBase.java [new file with mode: 0644]
xgboost/src/test/java/hivemall/TestUtils.java [new file with mode: 0644]
xgboost/src/test/java/hivemall/xgboost/XGBoostTrainUDTFTest.java [new file with mode: 0644]
xgboost/src/test/java/hivemall/xgboost/utils/DMatrixBuilderTest.java [new file with mode: 0644]