griffin.git
2 weeks ago[maven-release-plugin] prepare for next development iteration griffin-0.4.0-rc0 master
William Guo [Thu, 27 Dec 2018 12:22:12 +0000 (05:22 -0700)] 
[maven-release-plugin] prepare for next development iteration

2 weeks ago[maven-release-plugin] prepare release griffin-0.4.0 griffin-0.4.0
William Guo [Thu, 27 Dec 2018 12:21:30 +0000 (05:21 -0700)] 
[maven-release-plugin] prepare release griffin-0.4.0

2 weeks agoupdate to gitbox.apache.org in pom.xml
William Guo [Thu, 27 Dec 2018 11:39:00 +0000 (19:39 +0800)] 
update to gitbox.apache.org in pom.xml

Author: William Guo <guoyp@apache.org>

Closes #475 from guoyuepeng/update_scm_in_pom.

3 weeks agoupdate remote
William Guo [Wed, 26 Dec 2018 11:59:10 +0000 (19:59 +0800)] 
update remote

3 weeks agoMerge branch 'master' of https://github.com/apache/griffin
William Guo [Wed, 26 Dec 2018 11:39:46 +0000 (19:39 +0800)] 
Merge branch 'master' of https://github.com/apache/griffin

3 weeks agoImprove deployment manual guide
Eugene [Mon, 24 Dec 2018 08:16:51 +0000 (16:16 +0800)] 
Improve deployment manual guide

1.append detailed step's description
2.fix invalid stuffs

Author: Eugene <liujin@apache.org>

Closes #473 from toyboxman/pr-doc.

4 weeks agoMerge branch 'master' of https://github.com/apache/griffin
William Guo [Wed, 19 Dec 2018 06:26:25 +0000 (14:26 +0800)] 
Merge branch 'master' of https://github.com/apache/griffin

4 weeks agoMysql as storage profile
William Guo [Wed, 19 Dec 2018 05:01:57 +0000 (13:01 +0800)] 
Mysql as storage profile

Author: William Guo <guoyp@apache.org>

Closes #472 from guoyuepeng/mysql_as_storage_profile.

4 weeks agoMerge branch 'master' of https://github.com/apache/griffin
William Guo [Thu, 13 Dec 2018 23:48:54 +0000 (07:48 +0800)] 
Merge branch 'master' of https://github.com/apache/griffin

5 weeks agoFix asf GitHub sync issue 20181211
William Guo [Tue, 11 Dec 2018 00:13:22 +0000 (08:13 +0800)] 
Fix asf GitHub sync issue 20181211

Author: William Guo <guoyp@apache.org>

Closes #469 from guoyuepeng/fix_asf_github_sync_issue_20181211.

5 weeks agofix docs 469/head
William Guo [Mon, 10 Dec 2018 23:57:56 +0000 (07:57 +0800)] 
fix docs

5 weeks agodelete disclaimer
William Guo [Fri, 7 Dec 2018 07:55:50 +0000 (00:55 -0700)] 
delete disclaimer

5 weeks agoFix version mismatch
Eugene [Fri, 7 Dec 2018 07:25:22 +0000 (15:25 +0800)] 
Fix version mismatch

Revise build version, remove 'incubating' tag

Author: Eugene <liujin@apache.org>

Closes #467 from toyboxman/version.

5 weeks agoRemove incubator
William Guo [Thu, 6 Dec 2018 12:23:04 +0000 (20:23 +0800)] 
Remove incubator

Author: William Guo <guoyp@apache.org>

Closes #466 from guoyuepeng/remove_incubator.

6 weeks agoupdate merge script
William Guo [Wed, 5 Dec 2018 15:06:50 +0000 (23:06 +0800)] 
update merge script

6 weeks agoskip test csv data file
William Guo [Fri, 30 Nov 2018 05:18:03 +0000 (22:18 -0700)] 
skip test csv data file

Author: William Guo <guoyp@apache.org>

Closes #462 from guoyuepeng/fix_rat_license_20181130.

6 weeks agodoc improvement
Eugene [Fri, 30 Nov 2018 02:03:15 +0000 (19:03 -0700)] 
doc improvement

A minor update to give more introduction description

Author: Eugene <liujin@apache.org>

Closes #460 from toyboxman/doc.

7 weeks ago[GRIFFIN-213] Custom connector support
Nikolay Sokolov [Thu, 29 Nov 2018 08:00:47 +0000 (16:00 +0800)] 
[GRIFFIN-213] Custom connector support

Provide ability to extend batch and streaming data integrations
with custom user-provided connectors. Introduces new data connector
type, `CUSTOM`, parameterized with `class` property. Also adds support
for custom data connector enum on service side.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #456 from chemikadze/GRIFFIN-213.

8 weeks agoFix minor literal issues and improve doc content
Bolt [Wed, 21 Nov 2018 01:20:40 +0000 (09:20 +0800)] 
Fix minor literal issues and improve doc content

Author: Bolt <boltzjf@gmail.com>

Closes #459 from boltzjf/master.

8 weeks ago[GRIFFIN-212] Add matchedFraction in streaming accuracy metrics
Lionel Liu [Mon, 19 Nov 2018 08:57:57 +0000 (16:57 +0800)] 
[GRIFFIN-212] Add matchedFraction in streaming accuracy metrics

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #455 from bhlx3lyx7/util-update-0.3.

8 weeks agorevert
William Guo [Mon, 19 Nov 2018 08:55:15 +0000 (16:55 +0800)] 
revert

8 weeks agomerge_pr.py python 3 support
fanqiaoqing [Mon, 19 Nov 2018 08:04:12 +0000 (16:04 +0800)] 
merge_pr.py python 3 support

1. Import Request/urlopen/HTTPError from urllib.request to support python 3
2. The print statement has been replaced with a print() function

Author: fanqiaoqing <qiaoqing.fqq@gmail.com>

Closes #458 from IAmFQQ/merge_pr_python3_support.

2 months ago[SERVICE] JobInstace setJobInstanceIdAndUri, update instance.setAppU…
justACT [Fri, 16 Nov 2018 02:23:04 +0000 (10:23 +0800)] 
[SERVICE] JobInstace setJobInstanceIdAndUri, update instance.setAppU…

GRIFFIN-211 [Service] JobInstance appUrl error

Author: justACT <xiangrongchen19@163.com>

Closes #454 from justACT/newmaster.

2 months agoFixservicetests
William Guo [Thu, 15 Nov 2018 12:23:55 +0000 (20:23 +0800)] 
Fixservicetests

Author: William Guo <guoyp@apache.org>

Closes #453 from guoyuepeng/fixservicetests.

2 months ago[GRIFFIN-209] update param util, add UT for param util
Lionel Liu [Wed, 14 Nov 2018 14:33:29 +0000 (22:33 +0800)] 
[GRIFFIN-209] update param util, add UT for param util

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #452 from bhlx3lyx7/param-util.

2 months agoDefine griffin plain-vanilla hook.
Eugene [Fri, 9 Nov 2018 14:46:40 +0000 (22:46 +0800)] 
Define griffin plain-vanilla hook.

the purpose of hook is for integration with components outside. Griffin would offer information about internal task status.

Task: GRIFFIN-200

Author: Eugene <liujin@apache.org>
Author: William Guo <guoyp@apache.org>

Closes #444 from toyboxman/hook.

2 months ago[GRIFFIN-208] log exception details when it makes sense
Nikolay Sokolov [Tue, 30 Oct 2018 07:08:18 +0000 (15:08 +0800)] 
[GRIFFIN-208] log exception details when it makes sense

Sometimes error messages are not descriptive enough and do not allow
to pinpoint exact issue. Exception stack traces should make it easier
to troubleshoot issues.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #448 from chemikadze/GRIFFIN-208.

2 months ago[GRIFFIN-203] Plaintext mode" for measure creation
Arthur Gavlyukovskiy [Mon, 29 Oct 2018 01:55:44 +0000 (18:55 -0700)] 
[GRIFFIN-203] Plaintext mode" for measure creation

Created json/yaml measure creation flow:
![2018-10-24 21 41 44](https://user-images.githubusercontent.com/15277543/47476643-4aa37680-d7d6-11e8-9488-fe2e247617a1.gif)

Added ability to view measure as yaml:
![2018-10-24 21 42 23](https://user-images.githubusercontent.com/15277543/47476652-4d9e6700-d7d6-11e8-86f3-f35fdc44dda1.gif)

I did not add any input validation, for now only server side validation is working. I'm not quite sure if copying validation from service side is a good choice.

Author: Arthur Gavlyukovskiy <agavlyukovskiy@gmail.com>

Closes #446 from gavlyukovskiy/raw-measures.

2 months agoDocument compression, SSL and LDAP conf
Nikolay Sokolov [Mon, 29 Oct 2018 01:21:52 +0000 (18:21 -0700)] 
Document compression, SSL and LDAP conf

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #450 from chemikadze/patch-2.

2 months ago[GRIFFIN-194] single call to fetch all tables of all dbs
Nikolay Sokolov [Fri, 26 Oct 2018 09:07:08 +0000 (02:07 -0700)] 
[GRIFFIN-194] single call to fetch all tables of all dbs

Significantly improves opening speed for measure dialogs.
Previously taking 10+ seconds on huge schemas, with this call
it takes fraction of second to load.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #447 from chemikadze/GRIFFIN-194.

2 months agoSupport "cache" field on rules in service module
Nikolay Sokolov [Thu, 25 Oct 2018 15:19:12 +0000 (23:19 +0800)] 
Support "cache" field on rules in service module

Measure module exposes "cache" flag on rules, but it's not possible to store and schedule such a job in service module.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #442 from chemikadze/service-cache.

2 months agoUpdate deploy-guide.md
RichFy [Thu, 25 Oct 2018 14:50:35 +0000 (22:50 +0800)] 
Update deploy-guide.md

Update for psql command -p option explaination.

Author: RichFy <43976743+RichFy@users.noreply.github.com>

Closes #445 from RichFy/master.

2 months agoFix conventions in batch-sample
Nikolay Sokolov [Thu, 25 Oct 2018 14:41:14 +0000 (22:41 +0800)] 
Fix conventions in batch-sample

Bring conventions for enums in measure-batch-sample in sync with service module.
While measure accepts any case, service requires them capitalized.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #443 from chemikadze/patch-1.

2 months ago[GRIFFIN-207] LDAP login service improvements
Nikolay Sokolov [Thu, 25 Oct 2018 14:20:41 +0000 (22:20 +0800)] 
[GRIFFIN-207] LDAP login service improvements

  - allow non-CN usernames
  - allow disabling certificate checks
  - allow limiting set of login users
  - improve logging

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #441 from chemikadze/GRIFFIN-207.

2 months agoUpdate recommended ES index schema
Nikolay Sokolov [Mon, 22 Oct 2018 09:04:00 +0000 (17:04 +0800)] 
Update recommended ES index schema

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #440 from chemikadze/patch-1.

3 months agoComplement stuffs to docker guide.
Eugene [Tue, 16 Oct 2018 07:07:46 +0000 (15:07 +0800)] 
Complement stuffs to docker guide.

Author: Eugene <liujin@apache.org>

Closes #439 from toyboxman/doc/docker.

3 months agoupdate measure documents
Lionel Liu [Tue, 16 Oct 2018 06:19:44 +0000 (14:19 +0800)] 
update measure documents

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #438 from bhlx3lyx7/doc-update.

3 months ago[GRIFFIN-202] Show raw rules on UI
Nikolay Sokolov [Mon, 15 Oct 2018 09:06:31 +0000 (17:06 +0800)] 
[GRIFFIN-202] Show raw rules on UI

[GRIFFIN-202] Creates section on Measure Detail page, showing formatted JSON of the rule.

![2018-10-14 10 56 50](https://user-images.githubusercontent.com/867294/46920276-979a7800-cfa0-11e8-94c4-5ee4c58d5d46.gif)

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #437 from chemikadze/GRIFFIN-202.

3 months agoalways use apache griffin together as our mark
William Guo [Mon, 15 Oct 2018 05:53:08 +0000 (13:53 +0800)] 
always use apache griffin together as our mark

Author: William Guo <guoyp@apache.org>

Closes #436 from guoyuepeng/use_apache_griffin_as_trademark.

3 months ago[GRIFFIN-206] fix job timezone when creating from UI
Nikolay Sokolov [Mon, 15 Oct 2018 03:13:38 +0000 (11:13 +0800)] 
[GRIFFIN-206] fix job timezone when creating from UI

Demonstration of the problem:
```
> new Date()
Wed Oct 10 2018 11:30:48 GMT-0700 (Pacific Daylight Time)
> new Date().getTimezoneOffset()/60
7
```

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #435 from chemikadze/GRIFFIN-206.

3 months agoFix a little doc bug concerning measure part.
Eugene [Mon, 15 Oct 2018 03:08:32 +0000 (11:08 +0800)] 
Fix a little doc bug concerning measure part.

fix linguistic error and improve content

Author: Eugene <liujin@apache.org>

Closes #432 from toyboxman/doc/measure.

3 months ago[GRIFFIN-205] accuracy matched fraction
ashutak [Sat, 13 Oct 2018 10:30:48 +0000 (18:30 +0800)] 
[GRIFFIN-205] accuracy matched fraction

https://issues.apache.org/jira/browse/GRIFFIN-205

This pull request covers only batch dq type.
We need to decide is it worth to add "matched fraction" to streaming type.

Accuracy transformation tests added.

Author: ashutak <ashutak@griddynamics.com>

Closes #434 from ashutakGG/GRIFFIN-205-accuracy-matchedFraction.

3 months agoFix a bug in postman script
Eugene [Sat, 13 Oct 2018 03:51:23 +0000 (11:51 +0800)] 
Fix a bug in postman script

postman collection is not up to date, some APIs don't work well.
script has been refreshed in this patch by postman 2.1 standard.

Jira task: GRIFFIN-204

Author: Eugene <liujin@apache.org>

Closes #433 from toyboxman/doc/postman.

3 months ago[GRIFFIN-195] Don't list all tables from UI
Nikolay Sokolov [Mon, 8 Oct 2018 04:54:37 +0000 (12:54 +0800)] 
[GRIFFIN-195] Don't list all tables from UI

[GRIFFIN-195]
Initially, load only list of databases and list of table names.
Table information is loaded only if user clicks on specific table.
![2018-10-07 14 19 47](https://user-images.githubusercontent.com/867294/46587155-d0849b00-ca3c-11e8-91e4-655a16db6421.gif)

[GRIFFIN-196]
Search by table names.
![2018-10-07 14 20 11](https://user-images.githubusercontent.com/867294/46587161-d4182200-ca3c-11e8-99a7-4a4a55ca62a8.gif)

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #431 from chemikadze/GRIFFIN-193.

3 months agoFix doc issue.
Eugene [Mon, 8 Oct 2018 01:43:52 +0000 (09:43 +0800)] 
Fix doc issue.

dsl-guide describes two kinds of rules, Uniqueness and Distinctness. actually they are similiar rule, so we merely keep Uniqueness rule.

Author: Eugene <liujin@apache.org>

Closes #420 from toyboxman/doc/dsl-guide.

3 months agoremove duplicated cache settings
William Guo [Tue, 2 Oct 2018 15:07:56 +0000 (23:07 +0800)] 
remove duplicated cache settings

Author: William Guo <guoyp@apache.org>

Closes #430 from guoyuepeng/remove_dup_cache_in_travis.

3 months agoFix timeout failure in travis CI.
Eugene [Tue, 2 Oct 2018 15:01:05 +0000 (23:01 +0800)] 
Fix timeout failure in travis CI.

Build times out because no output was received.

'travis_wait mvn clean verify -q' will spawn a process to deal with mvn task
-q,--quiet     Quiet output - only show errors

-q probably leads no output caught by travis, so timeout failure happens

refer to:
https://docs.travis-ci.com/user/common-build-problems/#limitations-of-travis_wait

Author: Eugene <liujin@apache.org>

Closes #429 from toyboxman/build/tarvis.

3 months agoIncrease build silence timeout
Nikolay Sokolov [Mon, 1 Oct 2018 15:07:06 +0000 (23:07 +0800)] 
Increase build silence timeout

According to experiment, resolving dependencies takes longer
than default build timeout of 20 minutes. Increasing to 30.

[Experiment](https://travis-ci.org/apache/incubator-griffin/builds/435326777) shown that sometimes it takes more than 20 minutes to download all the dependencies and start executing tests. In that particular experiment it took 22 minutes in order to get to test execution.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #428 from chemikadze/increase-build-timeout.

3 months agoFix code bug.
Eugene [Mon, 1 Oct 2018 15:02:13 +0000 (23:02 +0800)] 
Fix code bug.

1.add java method description
2.reformat import statements violating Griffin's code rule https://github.com/apache/incubator-griffin/blob/master/griffin-doc/dev/code-style.md

Author: Eugene <liujin@apache.org>

Closes #426 from toyboxman/src/service.

3 months ago[GRIFFIN-197] Treat non-existing YARN app as FAILED
Nikolay Sokolov [Sun, 30 Sep 2018 07:09:16 +0000 (15:09 +0800)] 
[GRIFFIN-197] Treat non-existing YARN app as FAILED

This avoids jobs becoming stuck in UNKNOWN state on Service side.
Also, improves logging for YARN client errors.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #421 from chemikadze/GRIFFIN-197.

3 months agoFix case clauses
William Guo [Sat, 29 Sep 2018 10:21:42 +0000 (18:21 +0800)] 
Fix case clauses

Author: William Guo <guoyp@apache.org>

Closes #425 from guoyuepeng/fix_case_clauses.

3 months agoAdjust error msg when failing to create job
Li, Juan [Thu, 27 Sep 2018 09:28:04 +0000 (17:28 +0800)] 
Adjust error msg when failing to create job

Author: Li, Juan <sunshinejuanl@gmail.com>

Closes #423 from icesmartjuan/ui_createJobFailure_msg.

3 months agoMinor UI fixes
Nikolay Sokolov [Sat, 22 Sep 2018 13:56:01 +0000 (21:56 +0800)] 
Minor UI fixes

"Back" button text alignment
<img width="102" alt="screen shot 2018-09-21 at 6 02 19 pm" src="https://user-images.githubusercontent.com/867294/45912572-79de5680-bdd8-11e8-9acb-19670b7ca3d7.png">
<img width="86" alt="screen shot 2018-09-21 at 6 02 25 pm" src="https://user-images.githubusercontent.com/867294/45912573-7e0a7400-bdd8-11e8-9421-ef6dafc0d805.png">

footer line intersecting with content
<img width="175" alt="screen shot 2018-09-21 at 7 38 27 pm" src="https://user-images.githubusercontent.com/867294/45912578-92e70780-bdd8-11e8-8b39-2485047b422d.png">
<img width="162" alt="screen shot 2018-09-21 at 7 36 47 pm" src="https://user-images.githubusercontent.com/867294/45912580-97abbb80-bdd8-11e8-9a08-2b27b92a272b.png">

title rendering on sidebar
<img width="294" alt="screen shot 2018-09-21 at 7 29 53 pm" src="https://user-images.githubusercontent.com/867294/45912584-a4301400-bdd8-11e8-9ffe-e726dd912813.png">
<img width="323" alt="screen shot 2018-09-21 at 8 07 46 pm" src="https://user-images.githubusercontent.com/867294/45912670-0dfced80-bdda-11e8-87ae-b1985f55754e.png">

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #422 from chemikadze/ui-fixes.

3 months ago[GRIFFIN-198] Fix caching of incomplete results in HiveMetaStoreService
Nikolay Sokolov [Sat, 22 Sep 2018 13:46:32 +0000 (21:46 +0800)] 
[GRIFFIN-198] Fix caching of incomplete results in HiveMetaStoreService

Avoiding polluting cache with empty or null values.
Test caching behavior of HiveMetaStoreServiceImpl.
Verified that cache population in evictHiveCache() is not happening, added unittest.

This is only partial fix, getAllTable() calls still can return and cache incomplete results.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #419 from chemikadze/hive-caching.

3 months agoMake Rule table MySQL-compatible
Nikolay Sokolov [Fri, 21 Sep 2018 02:31:23 +0000 (10:31 +0800)] 
Make Rule table MySQL-compatible

OUT is reserved word in MySQL, so it should be quoted in queries.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #418 from chemikadze/fix-mysql.

3 months agoDeployment and compose doc improvements
Nikolay Sokolov [Thu, 20 Sep 2018 06:08:30 +0000 (14:08 +0800)] 
Deployment and compose doc improvements

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #417 from chemikadze/deployment-documentation.

3 months agoFix rest api bug in service component
Eugene [Thu, 20 Sep 2018 05:59:18 +0000 (13:59 +0800)] 
Fix rest api bug in service component

GET /api/v1/version returns mistaken release version 0.2.0 instead of 0.3.0

Author: Eugene <liujin@apache.org>

Closes #416 from toyboxman/src/service.

3 months agoSupport for HDP
Sparsh Singhal [Thu, 20 Sep 2018 05:48:30 +0000 (13:48 +0800)] 
Support for HDP

Added parameter for CISF protection in Ambari. Can be provided in a separate branch.

Author: Sparsh Singhal <sparsh.singhal@hashmapinc.com>

Closes #409 from SparshSinghalHM/patch-4.

4 months agoBring persist/sink documentation up to date
Nikolay Sokolov [Fri, 14 Sep 2018 07:16:42 +0000 (15:16 +0800)] 
Bring persist/sink documentation up to date

Another followup after 7b749ad72a78eb4244bcf47a80a52f2a6e3f222f

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #415 from chemikadze/sink-documentation.

4 months agofix startup failure bug
ahutsunshine [Fri, 14 Sep 2018 07:09:23 +0000 (15:09 +0800)] 
fix startup failure bug

Griffin starts up failure, as it  is could not resolve placeholder 'fs.defaultFS' in string value "${fs.defaultFS}". Add fs.defaultFS filed in application.properties  to fix the failure.

Author: ahutsunshine <ahutsunshine@gmail.com>

Closes #413 from ahutsunshine/master.

4 months agoFix API doc bug and complement some stuffs
Eugene [Fri, 14 Sep 2018 05:17:22 +0000 (13:17 +0800)] 
Fix API doc bug and complement some stuffs

Author: Eugene <liujin@apache.org>

Closes #412 from toyboxman/api.

4 months agoImprove document quality
Eugene [Fri, 14 Sep 2018 02:39:20 +0000 (10:39 +0800)] 
Improve document quality

Revise docker guide document and append more descriptions.

Author: Eugene <liujin@apache.org>

Closes #411 from toyboxman/doc/docker.

4 months agoFix download table bug
Li, Juan [Fri, 14 Sep 2018 02:32:45 +0000 (10:32 +0800)] 
Fix download table bug

Author: Li, Juan <sunshinejuanl@gmail.com>

Closes #410 from icesmartjuan/fix_ui_bug_2018905.

4 months agoupdate docker document for zookeeper image unaccessable in china
Lionel Liu [Fri, 14 Sep 2018 02:29:59 +0000 (10:29 +0800)] 
update docker document for zookeeper image unaccessable in china

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #408 from bhlx3lyx7/docker-doc.

4 months agoUpdated documentation for outputs
Nikolay Sokolov [Fri, 14 Sep 2018 02:21:57 +0000 (10:21 +0800)] 
Updated documentation for outputs

Documentation is out of sync with code after 7b749ad72a78eb4244bcf47a80a52f2a6e3f222f.
Updating measuring examples and format description accordingly.

Author: Nikolay Sokolov <chemikadze@gmail.com>

Closes #414 from chemikadze/out-documentation.

4 months ago[maven-release-plugin] prepare for next development iteration griffin-0.3.0-incubating-rc1
Lionel Liu [Thu, 30 Aug 2018 14:16:08 +0000 (07:16 -0700)] 
[maven-release-plugin] prepare for next development iteration

4 months ago[maven-release-plugin] prepare release griffin-0.3.0-incubating griffin-0.3.0-incubating
Lionel Liu [Thu, 30 Aug 2018 14:15:14 +0000 (07:15 -0700)] 
[maven-release-plugin] prepare release griffin-0.3.0-incubating

4 months agoIn order to improve readability, complement some stuffs into readme doc
Eugene [Thu, 30 Aug 2018 14:02:10 +0000 (22:02 +0800)] 
In order to improve readability, complement some stuffs into readme doc

Author: Eugene <liujin@apache.org>

Closes #407 from toyboxman/doc.

4 months agofix download disable items issue
Li, Juan [Thu, 30 Aug 2018 14:01:37 +0000 (22:01 +0800)] 
fix download disable items issue

fix the issue that one column is not disabled while in downloading

Author: Li, Juan <sunshinejuanl@gmail.com>

Closes #406 from icesmartjuan/fix_ui_bug.

4 months agoupdate gpg keys
William Guo [Thu, 30 Aug 2018 08:56:06 +0000 (16:56 +0800)] 
update gpg keys

Author: William Guo <guoyp@apache.org>

Closes #405 from guoyuepeng/update_gpg_keys.

4 months agofix license header
William Guo [Thu, 30 Aug 2018 07:54:03 +0000 (15:54 +0800)] 
fix license header

Author: William Guo <guoyp@apache.org>

Closes #404 from guoyuepeng/fix_service_header.

4 months agoupdate docker image source and docker compose file
Lionel Liu [Thu, 30 Aug 2018 07:28:55 +0000 (15:28 +0800)] 
update docker image source and docker compose file

we've transferred the docker image to docker hub of apachegriffin

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #403 from bhlx3lyx7/docker-update.

4 months agoset the fields in configuration classes private
grant-xuexu [Mon, 27 Aug 2018 02:05:44 +0000 (10:05 +0800)] 
set the fields in configuration classes private

in configuration case classes, get*** functions are defined to access to the fields, so for consistency, make the fields private

Author: grant-xuexu <grant.xuexu@gmail.com>

Closes #396 from grant-xuexu/configuration.

4 months agoupdate docker guide document
MintL617 [Mon, 27 Aug 2018 01:42:53 +0000 (09:42 +0800)] 
update docker guide document

for the latest docker image version

Author: MintL617 <mintlee617@gmail.com>

Closes #402 from MintL617/dev.

4 months agoFix UI bugs
Li, Juan [Fri, 24 Aug 2018 07:25:17 +0000 (15:25 +0800)] 
Fix UI bugs

1. In profiling, when selecting rule on UI, for "Enum detection Top5 count", it generates rule like:
{
    "rule": "source.desc, desc, count(*) AS count GROUP BY source.desc ORDER BY count DESC LIMIT 5",
    "dsl.type": "griffin-dsl",
    "dq.type": "PROFILING"
}
Actually, we need it to be like:
{
    "rule": "source.desc AS desc, count(*) AS count GROUP BY source.desc ORDER BY count DESC LIMIT 5",
    "dsl.type": "griffin-dsl",
    "dq.type": "PROFILING",
    "out": [
     {
            "type": "metric",
            "name": "desc_group",
            "flatten": "array"
        }
    ]
}
To flatten the group by metric as an array, and naming it as "${the group column name}_group". Then the metric will be like:
{"name":"prof_job","tmst":1534768500000,"value":{"desc_group":[{"desc": "aaa", "count": 3}, {"desc": "bbb", "count": 5}]}}

Please notice that there're 2 mistakes in current enum rule, one is the "rule" content, the other one is the lack of "out" field.

2. There's a bug in Dashboard page, when refreshing, it works good, and a metric chart is painted. But after select the measure drop-down box, the chart size is collapsed into a line, which is terrible.

3. In "partition configuration" pages of creating accuracy measure, the item "Time Zone" is blank by default, but in the same page of creating profiling measure, the item "Time Zone" has a default value, which should act the same as the accuracy measure creation page.

4. In "Select Models" page of creating profiling measure, when you select "Regular Expression Detection Count" as a rule for a string column, there will be a text box below, for regular expression. I think the UI is not bad so far.
However, it doesn't work in a right way. The rule it generates is like:
{
        "rule": "count(source.desc RLIKE '^[0-9]{4}$') AS `desc_regexcount`",
        "dsl.type": "griffin-dsl",
        "dq.type": "PROFILING"
    }
After test, it returns total count of the column, which means the statement "RLIKE" doesn't work. We need to change it like this:
{
        "rule": "count(source.desc) AS `desc_regexcount` WHERE source.desc RLIKE '^[0-9]{4}$'",
        "dsl.type": "griffin-dsl",
        "dq.type": "PROFILING"
    }
I've tested the latter, it works.

5. For "Download Miss Sample" page, the item list seems a litte strange, it would be better to list them in a table and hover with a hand pointer rather than a text cursor. Besides, sometimes it's slow for the downloading, I think it's better to tell user to wait after a click.

Author: Li, Juan <sunshinejuanl@gmail.com>

Closes #401 from icesmartjuan/fix_ui_bugs.

4 months agofix bugs of measure and service module for new json format
Lionel Liu [Fri, 24 Aug 2018 07:17:55 +0000 (15:17 +0800)] 
fix bugs of measure and service module for new json format

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #400 from bhlx3lyx7/json-update-new.

4 months agoInitialize java code checks rules for griffin
Eugene [Wed, 22 Aug 2018 01:08:46 +0000 (09:08 +0800)] 
Initialize java code checks rules for griffin

this config could be used by maven checkStyle plugin

Author: Eugene <toyboxman0001@163.com>

Closes #384 from toyboxman/checkstyle.

4 months agomake ui style consistent
William Guo [Mon, 20 Aug 2018 14:34:30 +0000 (22:34 +0800)] 
make ui style consistent

make ui module code style consistent

Author: William Guo <guoyp@apache.org>

Closes #398 from guoyuepeng/2018_08_20_make_ui_style_consistent.

4 months agoUpdate deploy-guide.md
Sparsh Singhal [Mon, 20 Aug 2018 14:32:45 +0000 (22:32 +0800)] 
Update deploy-guide.md

Updated document with new configuration files.

Author: Sparsh Singhal <sparsh.singhal@hashmapinc.com>

Closes #393 from SparshSinghalHM/patch-3.

5 months agoRemoved bug
Sparsh Singhal [Fri, 17 Aug 2018 22:58:10 +0000 (06:58 +0800)] 
Removed bug

Updated property to remove space.

Author: Sparsh Singhal <sparsh.singhal@hashmapinc.com>

Closes #397 from SparshSinghalHM/patch-2.

5 months agoFix bugs about code style violation against Griffin rules
Eugene [Thu, 16 Aug 2018 10:57:19 +0000 (18:57 +0800)] 
Fix bugs about code style violation against Griffin rules

Author: Eugene <toyboxman0001@163.com>

Closes #388 from toyboxman/wrap.

5 months agomerge pr382 and pr389
Li, Juan [Fri, 10 Aug 2018 10:34:24 +0000 (18:34 +0800)] 
merge pr382 and pr389

add out.dataframe.name in accuracy and profiling measure

Author: Li, Juan <sunshinejuanl@gmail.com>

Closes #391 from icesmartjuan/mergepr389382.

5 months agoupdate measure field to support new format and ut
ahutsunshine [Fri, 10 Aug 2018 10:28:48 +0000 (18:28 +0800)] 
update measure field to support new format and ut

1.update env_batch.json and env_streaming.json
2.Rule
- add "inDataFrameName" and "outDataFrameName", remove "name"
-  add "out" param array, move "metric", "record" param inside "out" array
DataSource
- add boolean field "baseline"
- change "cache" to "checkpoint"
DataConnector
- add "dataFrameName"
Measure
- add "sinks" string array
- update dqType from String to enum
JobServiceImpl
- change "persist" to "sinks"
-  compare literal string "hdfs" case insensitively

3.update measure ut and fix predicate ut bug

Author: ahutsunshine <ahutsunshine@gmail.com>

Closes #389 from ahutsunshine/master.

5 months agoModify measure module to support updated env and config json files format
Lionel Liu [Fri, 10 Aug 2018 10:16:30 +0000 (18:16 +0800)] 
Modify measure module to support updated env and config json files format

Some important modification in env.json
- "info.cache" -> "griffin.checkpoint"
- "persist" -> "sinks"
- "log" -> "console"
- "http" -> "elasticsearch"
- remove "cleaner"

Some important modiffication in dq.json
- add "baseline" in data source
- add "dataframe.name" in data connector
- in data source,"cache" -> "checkpoint"
- in rule and pre-proc rule, add "in.dataframe.name" and "out.dataframe.name", remove "name"
- in rule, add "out" param array, move "metric", "record" param inside "out" array
- add "sinks" string array as filter of sinks in env.json

Author: Lionel Liu <bhlx3lyx7@163.com>

Closes #382 from bhlx3lyx7/json-update.

5 months agofix container problem in macOs
jasonliaoxiaoge [Fri, 10 Aug 2018 02:58:07 +0000 (10:58 +0800)] 
fix container problem in macOs

fix container problem in macOs

Author: jasonliaoxiaoge <181276056@qq.com>

Closes #387 from jasonliaoxiaoge/master.

5 months agoTrim and reformat resource files
Eugene [Tue, 7 Aug 2018 08:30:22 +0000 (16:30 +0800)] 
Trim and reformat resource files

Author: Eugene <toyboxman0001@163.com>

Closes #386 from toyboxman/property.

5 months agoFix import violations by Griffin code check rules
Eugene [Tue, 7 Aug 2018 08:24:53 +0000 (16:24 +0800)] 
Fix import violations by Griffin code check rules

Author: Eugene <toyboxman0001@163.com>

Closes #385 from toyboxman/import.

5 months agoFix code style violations and improve quality
Eugene [Tue, 7 Aug 2018 08:20:49 +0000 (16:20 +0800)] 
Fix code style violations and improve quality

Author: Eugene <toyboxman0001@163.com>

Closes #383 from toyboxman/EnvConfig.

5 months ago[GRIFFIN-164][GRIFFIN-186][GRIFFIN-187] Profiling Re-factor + Regex/Empty String...
Spencer Hivert [Fri, 3 Aug 2018 07:35:41 +0000 (15:35 +0800)] 
[GRIFFIN-164][GRIFFIN-186][GRIFFIN-187] Profiling Re-factor + Regex/Empty String Support

We've been working away on Griffin here at Credit Karma, and we'd love to contribute back!

This PR tackles three separate tasks:

- [GRIFFIN-164](https://issues.apache.org/jira/browse/GRIFFIN-164): Regex Support
- [GRIFFIN-186](https://issues.apache.org/jira/browse/GRIFFIN-186): Create Profiling Measure Re-Factor
- [GRIFFIN-187](https://issues.apache.org/jira/browse/GRIFFIN-187): Empty String Support

The details for each of these tasks can be found in the JIRA tickets linked above!

Author: Spencer Hivert <spencer.hivert@creditkarma.com>

Closes #381 from spencer-hivert-ck/shivert/profiling-refactor-and-regex-support.

5 months agoFix java import style violations in codes by code style suggested by s…
Eugene [Fri, 3 Aug 2018 06:47:38 +0000 (14:47 +0800)] 
Fix java import style violations in codes by code style suggested by s…

…un/google

Author: Eugene <toyboxman0001@163.com>

Closes #380 from toyboxman/importStyle.

5 months agoFix code style bug in MeasureRepo
Eugene [Wed, 1 Aug 2018 23:59:28 +0000 (07:59 +0800)] 
Fix code style bug in MeasureRepo

Author: Eugene <toyboxman0001@163.com>

Closes #379 from toyboxman/MeasureRepo.

5 months agoFix code bugs and correct code style
Eugene [Wed, 1 Aug 2018 23:35:39 +0000 (07:35 +0800)] 
Fix code bugs and correct code style

Author: Eugene <toyboxman0001@163.com>

Closes #378 from toyboxman/SimpleCORSFilter.

5 months agoRemove unused method and clean codes
Eugene [Wed, 1 Aug 2018 23:30:58 +0000 (07:30 +0800)] 
Remove unused method and clean codes

Author: Eugene <toyboxman0001@163.com>

Closes #377 from toyboxman/measure.

5 months agoupdate dq job spec
William Guo [Tue, 31 Jul 2018 07:04:58 +0000 (15:04 +0800)] 
update dq job spec

Author: William Guo <guoyp@apache.org>

Closes #376 from guoyuepeng/update_griffin_job_spec.

5 months agoFix code bug and improve code quality
Eugene [Mon, 30 Jul 2018 01:00:34 +0000 (09:00 +0800)] 
Fix code bug and improve code quality

Author: Eugene <toyboxman0001@163.com>

Closes #375 from toyboxman/measure/fix.

5 months agoRemove duplicate jackson annotation
DongfangLu [Wed, 25 Jul 2018 12:54:04 +0000 (20:54 +0800)] 
Remove duplicate jackson annotation

Author: DongfangLu <donlu@ebay.com>
Author: louise <ayludongfang@163.com>

Closes #369 from ludongfang/master.

5 months agoFix doc bug and make it more clear and understandable
Eugene [Wed, 25 Jul 2018 12:32:11 +0000 (20:32 +0800)] 
Fix doc bug and make it more clear and understandable

Author: Eugene <toyboxman0001@163.com>

Closes #372 from toyboxman/doc/fix2.

5 months agoFix doc bug and improve doc readability
Eugene [Wed, 25 Jul 2018 12:17:21 +0000 (20:17 +0800)] 
Fix doc bug and improve doc readability

Author: Eugene <toyboxman0001@163.com>

Closes #371 from toyboxman/doc/fix1.

5 months agodelete single thread pool for hivemetastoreclient and print full log stack trace...
jasonliaoxiaoge [Tue, 24 Jul 2018 01:11:01 +0000 (09:11 +0800)] 
delete single thread pool for hivemetastoreclient and print full log stack trace in log4j

delete single thread pool for hivemetastoreclient and print full log stack trace in log4j

Author: jasonliaoxiaoge <181276056@qq.com>

Closes #370 from jasonliaoxiaoge/master.

5 months agoFix doc bug and improve readability of Griffin project introduction
Eugene [Tue, 24 Jul 2018 00:57:14 +0000 (08:57 +0800)] 
Fix doc bug and improve readability of Griffin project introduction

Author: Eugene <toyboxman0001@163.com>

Closes #368 from toyboxman/doc/fix2.