incubator-hugegraph-toolchain.git
7 months agochore: use cla assistant to support robot pr (#245)
imbajin [Mon, 21 Feb 2022 07:33:42 +0000 (15:33 +0800)] 
chore: use cla assistant to support robot pr (#245)

7 months agoBump postgresql from 42.2.5 to 42.3.3 (#244)
dependabot[bot] [Thu, 17 Feb 2022 05:23:10 +0000 (13:23 +0800)] 
Bump postgresql from 42.2.5 to 42.3.3 (#244)

Bumps [postgresql](https://github.com/pgjdbc/pgjdbc) from 42.2.5 to 42.3.3.
- [Release notes](https://github.com/pgjdbc/pgjdbc/releases)
- [Changelog](https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/pgjdbc/pgjdbc/compare/REL42.2.5...REL42.3.3)

---
updated-dependencies:
- dependency-name: org.postgresql:postgresql
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
9 months agoset draft to false at upload to release (#241) v0.12.0
coderzc [Tue, 28 Dec 2021 09:09:16 +0000 (17:09 +0800)] 
set draft to false at upload to release (#241)

9 months agoskip javadoc check and add maven badge (#240)
coderzc [Tue, 28 Dec 2021 08:22:56 +0000 (16:22 +0800)] 
skip javadoc check and add maven badge (#240)

9 months agoexclusion jackson from hive-exec (#239)
coderzc [Tue, 28 Dec 2021 07:35:03 +0000 (15:35 +0800)] 
exclusion jackson from hive-exec (#239)

9 months agorelease loader v0.12 (#238)
coderzc [Tue, 28 Dec 2021 07:09:05 +0000 (15:09 +0800)] 
release loader v0.12 (#238)

9 months agoBump httpclient to 4.5.13 and fix start with 'jdbc:' problem (#235)
coderzc [Wed, 22 Dec 2021 10:01:02 +0000 (18:01 +0800)] 
Bump httpclient to 4.5.13 and fix start with 'jdbc:' problem (#235)

9 months agoupload assert to release (#234)
coderzc [Tue, 21 Dec 2021 13:09:04 +0000 (21:09 +0800)] 
upload assert to release (#234)

9 months agoBump commons-compress from 1.19 to 1.21 (#224)
dependabot[bot] [Tue, 21 Dec 2021 07:43:05 +0000 (15:43 +0800)] 
Bump commons-compress from 1.19 to 1.21 (#224)

Bumps commons-compress from 1.19 to 1.21.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: imbajin <imbajin@users.noreply.github.com>
9 months agorelease maven package by actions (#232)
coderzc [Tue, 21 Dec 2021 06:13:21 +0000 (14:13 +0800)] 
release maven package by actions (#232)

9 months agoreform loader to adapt hugegraph-computer input (#230)
coderzc [Tue, 21 Dec 2021 03:46:25 +0000 (11:46 +0800)] 
reform loader to adapt hugegraph-computer input (#230)

9 months agochore: adopt mysql ci with action & upgrade ubuntu version (#231)
coderzc [Mon, 20 Dec 2021 08:20:12 +0000 (16:20 +0800)] 
chore: adopt mysql ci with action & upgrade ubuntu version (#231)

14 months agoImprove log info for files under dir (#223)
Linary [Thu, 29 Jul 2021 07:11:45 +0000 (15:11 +0800)] 
Improve log info for files under dir (#223)

15 months agochore: add issue template & auto stale issues and pr (#219)
imbajin [Fri, 2 Jul 2021 08:33:56 +0000 (16:33 +0800)] 
chore: add issue template & auto stale issues and pr (#219)

15 months agoUse github action to run ci (#217)
Linary [Wed, 23 Jun 2021 10:04:13 +0000 (18:04 +0800)] 
Use github action to run ci (#217)

Change-Id: Iffa3ac7005841e29dfcaaf620ccad17134be277e

15 months agoFix read time maybe negitive (#215)
Linary [Thu, 10 Jun 2021 12:43:32 +0000 (20:43 +0800)] 
Fix read time maybe negitive (#215)

Change-Id: Id79051845d71ef5001cee93014b2c6a638be821b

16 months agoFix: prase error when data props num is different (#211)
黄太白 [Fri, 21 May 2021 06:09:34 +0000 (14:09 +0800)] 
Fix: prase error when data props num is different (#211)

* fix #209

Co-authored-by: DoubleBabylol <18811639296@163.com>
16 months agoFix npe because missing edge mapping struct (#213)
Linary [Tue, 18 May 2021 02:54:03 +0000 (10:54 +0800)] 
Fix npe because missing edge mapping struct (#213)

Change-Id: I79e0c2a2d3e232cd2a5adb945214cbe3cbfe3b43

17 months agoLoad vertices before edges when check-vertex is true (#206)
Linary [Wed, 28 Apr 2021 03:08:46 +0000 (11:08 +0800)] 
Load vertices before edges when check-vertex is true (#206)

* Wait vertices finished before load edges
* Log parse and load progress info to track status
* Fix senstive pass

Change-Id: If1f6102afe6b1ed090520aecd52a9641606f446d

17 months agoFix rate inaccurate when print progress (#205)
Linary [Mon, 26 Apr 2021 06:29:24 +0000 (14:29 +0800)] 
Fix rate inaccurate when print progress (#205)

Change-Id: I7806987d76affb416f6f7daabd45a3999027c47c

17 months agoImprove some params in config and test (#204)
Linary [Fri, 16 Apr 2021 06:59:18 +0000 (14:59 +0800)] 
Improve some params in config and test (#204)

Change-Id: I474b8317582c5884fb2e92e14127bf123e1e4afe

17 months agoFix senstive mail address (#203)
Linary [Thu, 15 Apr 2021 10:04:22 +0000 (18:04 +0800)] 
Fix senstive mail address (#203)

Change-Id: Iadbaa5f6610c8f2dd2ae420a10f09856d7c0dc0a

17 months agoFix load time and rate of vertex/edge is inaccurate (#200)
Linary [Thu, 8 Apr 2021 02:49:42 +0000 (10:49 +0800)] 
Fix load time and rate of vertex/edge is inaccurate (#200)

* install HugeGraphServer from specified commit id

Change-Id: I6ec5e1d6d17d91d7f4f0e7db766d21056706f269

18 months agoBump guava from 28.1-jre to 29.0-jre (#199)
dependabot[bot] [Thu, 1 Apr 2021 02:55:06 +0000 (10:55 +0800)] 
Bump guava from 28.1-jre to 29.0-jre (#199)

Bumps [guava](https://github.com/google/guava) from 28.1-jre to 29.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
18 months agoUpdate struct.json (#198)
GitHubMaWu [Tue, 30 Mar 2021 07:07:14 +0000 (15:07 +0800)] 
Update struct.json (#198)

18 months agoCheck header not empty when user customized it (#195)
Linary [Thu, 11 Mar 2021 03:44:29 +0000 (11:44 +0800)] 
Check header not empty when user customized it (#195)

* update install-hdfs script

Change-Id: Iab8011d2235d29a79ad65d4cc380d801d08896ac

19 months agoUse array to optimize access perf in parse stage (#194)
Linary [Mon, 8 Feb 2021 12:57:56 +0000 (20:57 +0800)] 
Use array to optimize access perf in parse stage (#194)

Change-Id: I681255011c76067cef465f4687c0092dd08d2102

21 months agoAdd checkstyle plugin to check code style (#193)
Linary [Wed, 16 Dec 2020 08:48:03 +0000 (16:48 +0800)] 
Add checkstyle plugin to check code style (#193)

Change-Id: Ieb294cbe7d9fafa4ed468b965abe61135cafd50d

21 months agoFix date unequal since client and server has not same time zone (#192)
Linary [Wed, 16 Dec 2020 08:47:36 +0000 (16:47 +0800)] 
Fix date unequal since client and server has not same time zone (#192)

Change-Id: I708c3678df8ce64d5db49c40f8a3642738f00ca9

21 months agoInstall hugegraph from github instead of download link (#191)
Linary [Tue, 15 Dec 2020 02:54:11 +0000 (10:54 +0800)] 
Install hugegraph from github instead of download link (#191)

* Keep origin install-hugegraph-from-tar.sh

Change-Id: I2c051aeb81a60cfa0cd4418ca1361d7fd1ed758a

21 months agoCheck non-nullable keys before insert (#190)
Linary [Thu, 10 Dec 2020 14:54:00 +0000 (22:54 +0800)] 
Check non-nullable keys before insert (#190)

Change-Id: I4fa555da0bd0ff374f9bc7107731c89c33646145

22 months agoWrap exception when unset loading mode (#187) release-0.11.1 v0.11.1
Linary [Thu, 26 Nov 2020 11:18:08 +0000 (19:18 +0800)] 
Wrap exception when unset loading mode (#187)

Change-Id: Ic7d1e0880274339f653feabae68c5b6ccc4ebf7d

22 months agoFix bugs about jdbc url slash and trust store file path (#186)
Linary [Fri, 20 Nov 2020 11:25:22 +0000 (19:25 +0800)] 
Fix bugs about jdbc url slash and trust store file path  (#186)

fix #185

* Fix jdbc url has redudant slash
* fix specify trust file explicitly bug

Change-Id: Iada92d322fd428cd4826fc3622b655455f0b5bc1

22 months agoSupport https protocol used default truststore file and password (#183)
Linary [Thu, 19 Nov 2020 05:03:59 +0000 (13:03 +0800)] 
Support https protocol used default truststore file and password (#183)

Change-Id: Id0abd8f6e4ff161d4aad94ff10e25a9ee050d99b

22 months agoSupport enable kerberos authentication for HDFS (#176)
tmljob [Wed, 18 Nov 2020 11:42:40 +0000 (19:42 +0800)] 
Support enable kerberos authentication for HDFS (#176)

Co-authored-by: tmljob <tianminliang@encdata.cn>
22 months agoSupport parquet file by path and adapt primitive data type(#174)
tmljob [Wed, 18 Nov 2020 11:41:25 +0000 (19:41 +0800)] 
Support parquet file by path and adapt primitive data type(#174)

1. Read parquet file by path
2. Adapt INT96 etc. DataType

implement #172

Co-authored-by: tmljob@163.com <tianminliang@encdata.cn>
22 months agoUpgrade to version 0.11.0 for release (#173)
Linary [Fri, 6 Nov 2020 11:23:21 +0000 (19:23 +0800)] 
Upgrade to version 0.11.0 for release (#173)

Change-Id: I8c828790dfe910b8c683170c128db682fce797e6

22 months agoAdapt snappy-raw compression read (#119)
Linary [Wed, 4 Nov 2020 09:33:38 +0000 (17:33 +0800)] 
Adapt snappy-raw compression read (#119)

* adapt afs
* comment afs dependency

Change-Id: Ie38ba73573d12edcfd7eaac22a3b041c781dc7f6

2 years agoSome improve for stopping load (#170)
Linary [Wed, 30 Sep 2020 03:06:03 +0000 (11:06 +0800)] 
Some improve for stopping load (#170)

Change-Id: I78ab4e61b51641717ab02b240077803840bc1650

2 years agoSet loading mode before load and reset after load (#169)
Linary [Tue, 29 Sep 2020 06:12:41 +0000 (14:12 +0800)] 
Set loading mode before load and reset after load (#169)

Change-Id: Ic8edcea08178db295313c9c40e173a0e6438c6e8

2 years agoModify struct_hdfs.json in testSourceOrTargetPrimaryValueEmpty (#167)
Linary [Fri, 31 Jul 2020 11:11:44 +0000 (19:11 +0800)] 
Modify struct_hdfs.json in testSourceOrTargetPrimaryValueEmpty (#167)

Change-Id: I971c18bd766c84b328d6d8e62fb96bbfb9b944e5

2 years agoSkip vertex whose primary values are all empty (#166)
Linary [Thu, 30 Jul 2020 07:34:24 +0000 (15:34 +0800)] 
Skip vertex whose primary values are all empty (#166)

* Skip vertex whose primary values are all empty
* Let partial primary value empty also skipped

Implement #165

Change-Id: Ie9e1cc1f4141f9f63f57734faf36caefacef9691

2 years agoSupport timestamp as date format (#164)
Linary [Wed, 29 Jul 2020 12:15:07 +0000 (20:15 +0800)] 
Support timestamp as date format (#164)

* Support timestamp as date format
* handle some https test cases
* upgrade dependency of hugegraph-client

Change-Id: I1b100b52f762046e3bdfa2c1df44b66a223758b3

2 years agoAdd client https method (#161)
shzcore [Mon, 27 Jul 2020 14:03:23 +0000 (22:03 +0800)] 
Add client https method (#161)

2 years agoSome small optimization (#163)
Linary [Fri, 3 Jul 2020 02:51:01 +0000 (10:51 +0800)] 
Some small optimization (#163)

Change-Id: I9bd2254629b0bab3a7dfc2622c7435fd8f1b30f5

2 years agoSupport specify the max read lines (#159)
Linary [Wed, 17 Jun 2020 09:52:58 +0000 (17:52 +0800)] 
Support specify the max read lines (#159)

* Support specify the max read lines
* add suffix L for long

Change-Id: I0efcefd651fdb4527ccb9be725e258b3ecd9a44d

2 years agoSome improve for adapting hubble (#151)
Linary [Mon, 15 Jun 2020 06:59:50 +0000 (14:59 +0800)] 
Some improve for adapting hubble (#151)

* Some improve for adapting hubble
* config mvn release options

Change-Id: I2fd894471d2b97f6b4e6b603794a895007039696

2 years agoSupport read parquet file (#153)
Linary [Mon, 25 May 2020 04:11:23 +0000 (12:11 +0800)] 
Support read parquet file (#153)

* Support read parquet file
* Add nonfilteredfileextensions to avoid parquet file changed

Change-Id: I099067adc27e584c235a6121a66af66f2c449ad8

2 years agoImproved some usability issues (#147)
Linary [Thu, 9 Apr 2020 10:06:52 +0000 (18:06 +0800)] 
Improved some usability issues (#147)

* Improved some usability issues
* Adapt for hubble backend
* upgrade client version

Change-Id: I8b1a8a6ffd170712ec85d506181f5f44fd4c182c

2 years agoSupport config username in load options (#146)
Linary [Sun, 5 Apr 2020 11:53:41 +0000 (19:53 +0800)] 
Support config username in load options (#146)

Change-Id: I609a6e6b3be366d8ee5953879004d85fb2c33ab9

2 years agoRefactored load mapping file version 2.0 (#121)
Linary [Mon, 30 Mar 2020 03:55:57 +0000 (11:55 +0800)] 
Refactored load mapping file version 2.0 (#121)

* Let parse run in parallel

Change-Id: I40deb5414b98069b5f201e439d1e524e8c3aee45

* Let LoadContext as global singleton

Change-Id: I50c4333e75c403b155eb915a564c30b6e8152737

* Let schema cache init before load

Change-Id: I7188682dedebce1bee369d90e388482223bf43db

* Add ReadException

Change-Id: I931767cdc40a2250b546a9148f199efea10973b1

* Support skip vertex and edge with empty id

Change-Id: I6316724c9f02f9e2b961c722cc1311b503951ba2

implement #35
implement #85
implement #102

* Execute parse task in series mode

Change-Id: If649a06d92d2fde3e640c26e03873acde7dbcae1

* rebase and arrange code

Change-Id: I376e3c1c000d0d5042e49f3857e9578d402b2b4f

* add option used to load failure data

Change-Id: I840820686565eeee91cfb1decc561a98068c5423

* tiny improve

Change-Id: I066cb0638cc022c9b616ae55bd6aa6f2fd7c790a

* support unfold column to multiple vertices or edges

Change-Id: Id21f23da53c77d28b45507ba0cbc9011f2db80df

* Add some unfold tests and tiny improve

Change-Id: I1294978792ec4751ccf825ed9999717aaa70cee0

* Add mapping-convert.sh

Change-Id: I15610ac2640f2893ca353dc5d0ba89322992fd47

* Let failure mode force to scan entire error file

Change-Id: I411e04571c55166b39244b947cc3df9494e964b2

* tiny improve

Change-Id: I186a06b342d77e9e031e1ac13dc4aa819b8d859a

2 years agoSupport to clear schema before loading (#140)
Linary [Fri, 20 Mar 2020 06:21:52 +0000 (14:21 +0800)] 
Support to clear schema before loading (#140)

Change-Id: Icc8c096e6c1539370ccfcd83b4a9753ab03040b1

2 years agoSupport id field stored as property (#143)
Linary [Thu, 19 Mar 2020 09:17:46 +0000 (17:17 +0800)] 
Support id field stored as property (#143)

Fix #133

Change-Id: I6f91c8073029d430a934e68f69f3167b182720f7

2 years agoEnsure primary key columns valid or participate field mapping (#141)
Linary [Wed, 11 Mar 2020 06:41:35 +0000 (14:41 +0800)] 
Ensure primary key columns valid or participate field mapping (#141)

Change-Id: I7bd1cc6c6333032b0ba1f2460a6fe25696fab420

2 years agoEncoded number and date primary values (#136)
Linary [Mon, 24 Feb 2020 05:15:00 +0000 (13:15 +0800)] 
Encoded number and date primary values (#136)

Change-Id: I797963f9977cf8bdde81037b29e79ba81e566c93

2 years agoUse SimpleFlatMapper csv parser (#124)
Linary [Wed, 5 Feb 2020 08:30:42 +0000 (16:30 +0800)] 
Use SimpleFlatMapper csv parser (#124)

Change-Id: I852f8438a5d9d1e7a31a4bf9f36366d13923ba4a

2 years agoAdd an shell tool used to convert utf8-bom to utf8 (#128)
Linary [Mon, 3 Feb 2020 09:17:06 +0000 (17:17 +0800)] 
Add an shell tool used to convert utf8-bom to utf8 (#128)

Change-Id: If9bbcfdba9a78b28a5034c17916ab2eacefb2345

2 years agoSplit num threads for batch and single insert (#120)
Linary [Mon, 30 Dec 2019 14:35:16 +0000 (22:35 +0800)] 
Split num threads for batch and single insert (#120)

Change-Id: Ib00182949aa3ce74a5c7170f537e8463b03d0d7e

2 years agoSupport to load orc compression file (#113)
Linary [Mon, 30 Dec 2019 03:23:03 +0000 (11:23 +0800)] 
Support to load orc compression file (#113)

implement #45

Change-Id: Ia96cf31e8c8dd5c935da7b93789d63c1aba6afda

Co-authored-by: Dominic <283157054@qq.com>
2 years agoLet print insert error just once time to avoid too much output (#118)
Linary [Tue, 24 Dec 2019 09:36:29 +0000 (17:36 +0800)] 
Let print insert error just once time to avoid too much output (#118)

Change-Id: I947c86b38f2aaf1736dc66b1a6a303ad234aa21a

2 years agoLet single edge insert support options.checkVertex (#117)
Linary [Thu, 12 Dec 2019 06:31:39 +0000 (14:31 +0800)] 
Let single edge insert support options.checkVertex (#117)

Change-Id: Id559e0d1e5ce4dc6f54b68646e1883a4fa6fb9e9

2 years agoSupport customized timezone for date (#107)
Linary [Thu, 5 Dec 2019 13:28:53 +0000 (21:28 +0800)] 
Support customized timezone for date (#107)

Change-Id: I9814239e101e1424fc6d247a9984fcaaedbcca5c

2 years agoFix NPE when json file reload (#112)
Linary [Thu, 5 Dec 2019 10:13:14 +0000 (18:13 +0800)] 
Fix NPE when json file reload (#112)

Fix #111

2 years agoSupport read hadoop conf file (#105)
Dominic [Tue, 26 Nov 2019 08:51:58 +0000 (16:51 +0800)] 
Support read hadoop conf file (#105)

implement #104

2 years agoFix line retainAll doesn't modify names and values array (#110)
Linary [Tue, 26 Nov 2019 07:46:26 +0000 (15:46 +0800)] 
Fix line retainAll doesn't modify names and values array (#110)

Change-Id: I383b3b20a162816098aa8ee8468e8255b3553fdc

2 years agoUpgrade version to 0.10.0 (#103)
Linary [Fri, 8 Nov 2019 06:08:58 +0000 (14:08 +0800)] 
Upgrade version to 0.10.0 (#103)

Change-Id: If74cc9fd3e9cf132b3c4fc3f9ff0c135d3077634

2 years agoSupport skip struct to avoid delete it (#101)
Linary [Thu, 7 Nov 2019 08:26:26 +0000 (16:26 +0800)] 
Support skip struct to avoid delete it (#101)

implement #100

Change-Id: Ie7fb4a11e17e957b111dddb972845f444a5e8053

2 years agoFix list or set property update failed (#99)
Linary [Wed, 6 Nov 2019 10:14:19 +0000 (18:14 +0800)] 
Fix list or set property update failed (#99)

Change-Id: Ia4d307776b86c6bcbe2b06957b4cf4cb921d9c3d

2 years agoLog failure record to compensate offset ahead problem (#87)
Linary [Wed, 16 Oct 2019 11:42:14 +0000 (19:42 +0800)] 
Log failure record to compensate offset ahead problem (#87)

Change-Id: Ibba36ae57444ded2f39261512df6704725b486d7

2 years agoSupport batch update element properties with strategies (#97)
imbajin [Wed, 9 Oct 2019 09:09:23 +0000 (17:09 +0800)] 
Support batch update element properties with strategies (#97)

The strategy in JSON is case sensitive now, wait for jackthon update to V2.9+ to support case-insensitive original

2 years agoSupport uuid idstrategy (#98)
Linary [Wed, 9 Oct 2019 02:44:27 +0000 (10:44 +0800)] 
Support uuid idstrategy (#98)

Change-Id: If7bd8a59eccf4cfb6d138a7acba1923f912dbc2c

3 years agoImprove some code (#96)
Linary [Tue, 3 Sep 2019 09:39:10 +0000 (17:39 +0800)] 
Improve some code (#96)

Change-Id: Ic7addbf75b324c931aeeca7f244e993547d136d1

3 years agoSupport unsigned long property (#91)
Linary [Thu, 29 Aug 2019 11:19:34 +0000 (19:19 +0800)] 
Support unsigned long property (#91)

Change-Id: Iae2aa1e6d4f1b765d902fda6613d6143b3590f69

3 years agoFix boolean property parse failed (#92)
Linary [Tue, 27 Aug 2019 11:27:24 +0000 (19:27 +0800)] 
Fix boolean property parse failed (#92)

Change-Id: I539c3058cb9c4b850f5e53fd00872a32b76fe817

3 years agoOptimize processing logic for line with extra or missing columns (#93)
Linary [Mon, 26 Aug 2019 12:03:11 +0000 (20:03 +0800)] 
Optimize processing logic for line with extra or missing columns (#93)

Change-Id: If48cc313cf0073329916e1d33f92b1ae749b1d12

3 years agoFix empty table hasNext return true (#90)
Linary [Mon, 19 Aug 2019 03:18:35 +0000 (11:18 +0800)] 
Fix empty table hasNext return true (#90)

Change-Id: I6449b93a23c01ed804c71c650d9f8712b26a1af7

3 years agoSupport read header in different releation database (#79)
Linary [Fri, 9 Aug 2019 09:07:48 +0000 (17:07 +0800)] 
Support read header in different releation database (#79)

Implement #77
Change-Id: I83bb8ea8652463ceeba520ae3f4a76d169ecae83

3 years agoImprove some perf on reader, parser and builder (#84)
Linary [Mon, 5 Aug 2019 08:39:59 +0000 (16:39 +0800)] 
Improve some perf on reader, parser and builder (#84)

Change-Id: Ida4f3bba6c79c0a53e3ef35826648b7bfd17bb78

3 years agoFix acquired semaphores num error when wait task finished (#86)
Linary [Fri, 2 Aug 2019 09:08:13 +0000 (17:08 +0800)] 
Fix acquired semaphores num error when wait task finished (#86)

Change-Id: I112261828af7267ba7463461153486af3ea97d4f

3 years agoSupport record load progress for resume loading (#70)
Linary [Mon, 1 Jul 2019 13:02:56 +0000 (21:02 +0800)] 
Support record load progress for resume loading (#70)

Implement #69

Change-Id: I39578958f0915f42840bf2789369df647b4867a0

3 years agoSupport filter files by file name suffix (#82)
Linary [Wed, 26 Jun 2019 07:10:06 +0000 (15:10 +0800)] 
Support filter files by file name suffix (#82)

Implement #44

Change-Id: I25484c390015daacd907ea102c31c1f8aaa8f495

3 years agoEnhance summary info (#80)
imbajin [Thu, 20 Jun 2019 03:47:11 +0000 (11:47 +0800)] 
Enhance summary info (#80)

1. add parserSuccess to comfirm data integrity
2. add parserSpeed to get performance results directly
3. record the import speed of each data source in the log for comparison

3 years agoAdd http-connection options (#81)
imbajin [Tue, 18 Jun 2019 13:28:40 +0000 (21:28 +0800)] 
Add http-connection options (#81)

3 years agoFix parser exception cannot be caught (#74)
imbajin [Mon, 17 Jun 2019 05:13:07 +0000 (13:13 +0800)] 
Fix parser exception cannot be caught (#74)

3 years agoAdd an introduction about the supported data sources (#78)
Linary [Fri, 14 Jun 2019 03:56:45 +0000 (11:56 +0800)] 
Add an introduction about the supported data sources (#78)

Change-Id: I63c454f8cdafa922174a952931e171b63c6b2a4e

3 years agoSupport value mapping (#67)
Linary [Tue, 7 May 2019 10:11:15 +0000 (18:11 +0800)] 
Support value mapping (#67)

Change-Id: I39cad735f176cc772dd51c48f4d61537bbcf1c51

3 years agoSupport customized list element delimiter in text file (#66)
Linary [Thu, 25 Apr 2019 07:55:29 +0000 (15:55 +0800)] 
Support customized list element delimiter in text file (#66)

Implement #65
Change-Id: I9c20483c38133a04ce20e9a0843a2dce98d1ec73

3 years agoSupport "selected" in JSON config of data source (#62)
Linary [Mon, 22 Apr 2019 13:12:59 +0000 (21:12 +0800)] 
Support "selected" in JSON config of data source (#62)

Implement #61
Change-Id: I9fe630ebd823d678e1d2d95d497d91f1fdf5760a

3 years agoFix nullable key in mapping field can't work (#64)
Linary [Mon, 22 Apr 2019 08:25:38 +0000 (16:25 +0800)] 
Fix nullable key in mapping field can't work (#64)

Fix #63

Change-Id: I87d47ef14334a8326ce96e4def4571a0ed31e5fd

3 years agoUpgrade version to 0.9.0 (#60)
Linary [Fri, 19 Apr 2019 10:18:46 +0000 (18:18 +0800)] 
Upgrade version to 0.9.0 (#60)

Change-Id: Iee025b4e6700bd61936f5ccea9a02608c1fb8c83

3 years agoreader: improve some error messages (#52)
Jermy Li [Tue, 9 Apr 2019 07:35:15 +0000 (02:35 -0500)] 
reader: improve some error messages (#52)

Change-Id: I974377a676908a9f46c26c45d7a41c7e913298b0

3 years agoremove unused code (#53)
Jermy Li [Tue, 9 Apr 2019 07:26:23 +0000 (02:26 -0500)] 
remove unused code (#53)

Change-Id: I120fe8119b53c484b45a02e84e692424fdece599

3 years agoadd serial id to XxException class (#54)
Jermy Li [Tue, 9 Apr 2019 07:16:35 +0000 (02:16 -0500)] 
add serial id to XxException class (#54)

Change-Id: I00bb4a738333b95312403c5bd5cde954487c0d5b

3 years agoThe import Unmodifiable is never used (#51)
浩宇天罡 [Tue, 9 Apr 2019 06:57:51 +0000 (14:57 +0800)] 
The import Unmodifiable is never used (#51)

The import org.apache.commons.collections.Unmodifiable is never used

3 years agoExample problem fix (#57)
Linary [Tue, 9 Apr 2019 06:50:39 +0000 (14:50 +0800)] 
Example problem fix (#57)

Change-Id: I52b3257f5020cd39689654d5a2d473efcb4cb907

3 years agoUpgrade and temporarily comment vulnerable dependency (#48)
Linary [Wed, 27 Mar 2019 12:28:20 +0000 (20:28 +0800)] 
Upgrade and temporarily comment vulnerable dependency (#48)

Change-Id: Iaebd63883e0878c0c1a822e8e4f9a6bf466c2b5d

3 years agoAdd an option 'token' for authentication (#46)
Linary [Fri, 22 Mar 2019 09:20:40 +0000 (17:20 +0800)] 
Add an option 'token' for authentication (#46)

Change-Id: Id2d748f83ce3c52dc961ba32b044dff6e40b6615

3 years agoimprove url regex regex (#47)
zhoney [Wed, 20 Mar 2019 09:02:41 +0000 (17:02 +0800)] 
improve url regex regex (#47)

fixed: #37

Change-Id: Ieafb22a7805ecb834a7d412a770b115c6bbd98f9

3 years agoSupport config regex to skip specified line (#43)
Linary [Mon, 18 Mar 2019 07:58:52 +0000 (15:58 +0800)] 
Support config regex to skip specified line (#43)

Implement #42

Change-Id: I08fd43a65706f9a402b2ac328a3a5c034b8445df

3 years agoSupport load data from HDFS and relational database (#14)
Linary [Mon, 18 Mar 2019 06:28:06 +0000 (14:28 +0800)] 
Support load data from HDFS and relational database (#14)

Support load data from HDFS and relational database

1. Refactoring code for unit tests
- Add test for ordinary text file on hdfs
- Add test for compressed files
2. Add tests for jdbc(mysql)
3. nstall mysql when profile is jdbc, remove unused test suite
4. Support pass 'fs.defaultFS' to connect remote hadoop

Change-Id: Idbae121701a97d0b7abce903621842f14877bd45