helix.git
4 months agoUpgrade helix-front to Angular 8.2 (#2089)
Micah Stubbs [Thu, 2 Jun 2022 00:09:33 +0000 (17:09 -0700)] 
Upgrade helix-front to Angular 8.2 (#2089)

* ng update @angular/cli --from=5.2.11 --to=6 --migrate-only

* npm i -D rxjs-tslint, add rxjs specific linting rules, add lint:tslint alternative linting script

* npm install @angular/cdk@6.1.0

* npm install @angular/{animations,cdk,common,compiler,compiler-cli,core,forms,material,platform-browser,platform-browser-dynamic,platform-server,router}@6.1.0

* npm i ajv@6.9.1

* npm i codelyzer@6.0.1

* npm i ngx-clipboard@11.1.5

* npm i tsickle@0.32.1

* restore rxjs-compat@6.0.0-rc.0 to fix rxjs issue at npm run build

* register proxy.conf.json in ng serve builder in angular.json

* WIP state with new Reactive Form

* Revert "WIP state with new Reactive Form"

This reverts commit f9e2e375773b8e4428e2a887234d482b591682bb.

* upgrade to typescript@^3.2.4

* pin typescript at 3.2.4

* npm i tsickle@0.34.2

* npm i @swimlane/ngx-datatable@14.0.0

* npm i @swimlane/ngx-graph@6.0.0-rc.2

* manually upgrade to Angular 8.2.14 and associated major deps

* fix new TypeScript errors

* update bundle target to ES2015 since Angular 8 now automatically creates an ES5 compat build too

* @ViewChild {static: true}

* NG_DISABLE_VERSION_CHECK=1 npx @angular/cli@8 update @angular/material@8 --force

* use @angular/core@8.2.3 and friends, fix bad upgrade script that jumped ahead to v9

* npm i typescript@3.5.3

* npm i zone.js@~0.9.1

* npm i tsickle@0.37.0

* npm i @angular/flex-layout@8.0.0-beta.27

* npm i @swimlane/ngx-datatable@16.0.0

* npm i @ngtools/webpack@9.0.0-next.7

* npm uninstall node-sass && npm i sass

* nvm use 10.9.0 && npm i

* manually update material imports, use deep imports

* npm i -D @angular-devkit/build-angular@0.803.29

* npm i ngx-clipboard@12.1.2

* npm i @swimlane/ngx-charts@13.0.4

* npm i @swimlane/ngx-graph@7.0.0

* npm i @swimlane/ngx-datatable@16.0.3

* npm i @swimlane/ngx-datatable@16.1.0

* update ngx-datatable style paths: no longer a release folder

* add trailing newline to tsconfig

* Upgrade helix-front to Angular 8.2 Fix #2088

    Fix security vulnerabilities in helix-front dependencies.
    Upgrade helix-front dependencies to improve contributor productivity.

4 months agoUpgrade helix-front to Angular 7.2 (#2087)
Micah Stubbs [Wed, 1 Jun 2022 21:22:36 +0000 (14:22 -0700)] 
Upgrade helix-front to Angular 7.2 (#2087)

Fix security vulnerabilities in helix-front dependencies.
Upgrade helix-front dependencies to improve contributor productivity.

4 months agoupgrade helix-front to Angular 6.1 (#2083)
Micah Stubbs [Wed, 1 Jun 2022 17:30:34 +0000 (10:30 -0700)] 
upgrade helix-front to Angular 6.1 (#2083)

Fix security vulnerabilities in helix-front dependencies.
Upgrade helix-front dependencies to improve contributor productivity.

4 months agoFix static instance bug in DistViewAggregatorStateModel (#2124)
Qi (Quincy) Qu [Tue, 31 May 2022 19:35:57 +0000 (12:35 -0700)] 
Fix static instance bug in DistViewAggregatorStateModel (#2124)

Change to use non-static reference in statemodel

4 months agoClean up after TestAuthValidator -- fix TestClusterAccessor (#2097)
xyuanlu [Tue, 31 May 2022 18:16:27 +0000 (11:16 -0700)] 
Clean up after TestAuthValidator --  fix TestClusterAccessor (#2097)

Clean up after TestAuthValidator --  fix TestClusterAccessor

4 months agoReduce helix controller log and minor code improve (#2102)
Qi (Quincy) Qu [Fri, 27 May 2022 17:11:27 +0000 (10:11 -0700)] 
Reduce helix controller log and minor code improve (#2102)

Turn down a few log level to DEBUG and reduce duplicated logs

4 months agoCleanup ivy file in helix-view-aggregator (#2110)
Qi (Quincy) Qu [Fri, 27 May 2022 17:10:58 +0000 (10:10 -0700)] 
Cleanup ivy file in helix-view-aggregator (#2110)

Remove unused code block

4 months agoupgrade helix-front from Angular 5.1 to 5.2 (#2108)
Micah Stubbs [Thu, 26 May 2022 17:08:42 +0000 (10:08 -0700)] 
upgrade helix-front from Angular 5.1 to 5.2 (#2108)

Incremental progress towards the goal of upgrading helix-front to the current version of Angular.

4 months agoFix incompatible issue for clusterConfig mapfields disabledInstances (#2100)
xyuanlu [Wed, 18 May 2022 20:16:34 +0000 (13:16 -0700)] 
Fix incompatible issue for clusterConfig mapfields disabledInstances (#2100)

Fix incompatible issue for clusterConfig mapfields

4 months agoremove missing helix-front tslint linting rules to fix warnings (#2077)
Micah Stubbs [Tue, 17 May 2022 17:56:28 +0000 (10:56 -0700)] 
remove missing helix-front tslint linting rules to fix warnings (#2077)

remove missing helix-front tslint linting rules to fix warnings

4 months agospecify known working nodejs and npm versions in package.json (#2074)
Micah Stubbs [Tue, 17 May 2022 17:55:56 +0000 (10:55 -0700)] 
specify known working nodejs and npm versions in package.json (#2074)

helix-front: specify known working nodejs and npm versions in package.json

4 months agoDefine development process
Junkai Xue [Thu, 12 May 2022 05:27:11 +0000 (22:27 -0700)] 
Define development process

4 months agoUpdate pom and update ivy files
Junkai Xue [Thu, 12 May 2022 05:05:23 +0000 (22:05 -0700)] 
Update pom and update ivy files

4 months agoRelease note for 1.0.4
Junkai Xue [Thu, 12 May 2022 05:03:25 +0000 (22:03 -0700)] 
Release note for 1.0.4

4 months ago[maven-release-plugin] prepare for next development iteration
Junkai Xue [Thu, 12 May 2022 03:27:41 +0000 (20:27 -0700)] 
[maven-release-plugin] prepare for next development iteration

4 months ago[maven-release-plugin] prepare release helix-1.0.4
Junkai Xue [Thu, 12 May 2022 03:27:26 +0000 (20:27 -0700)] 
[maven-release-plugin] prepare release helix-1.0.4

4 months agoEnable helix-front for release
Junkai Xue [Thu, 12 May 2022 02:58:00 +0000 (19:58 -0700)] 
Enable helix-front for release

4 months agoRevert "[maven-release-plugin] prepare release helix-1.0.4"
Junkai Xue [Thu, 12 May 2022 02:56:12 +0000 (19:56 -0700)] 
Revert "[maven-release-plugin] prepare release helix-1.0.4"

This reverts commit ef684139187f2a33c1941e4075e76f8c2c104746.

4 months agoRevert "[maven-release-plugin] prepare for next development iteration"
Junkai Xue [Thu, 12 May 2022 02:55:55 +0000 (19:55 -0700)] 
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 013042e97b82e4d7a4556988b310379f20955987.

4 months ago[maven-release-plugin] prepare for next development iteration
Junkai Xue [Thu, 12 May 2022 01:27:06 +0000 (18:27 -0700)] 
[maven-release-plugin] prepare for next development iteration

4 months ago[maven-release-plugin] prepare release helix-1.0.4
Junkai Xue [Thu, 12 May 2022 01:26:52 +0000 (18:26 -0700)] 
[maven-release-plugin] prepare release helix-1.0.4

4 months agoFix release URL
Junkai Xue [Thu, 12 May 2022 01:18:48 +0000 (18:18 -0700)] 
Fix release URL

4 months agoRefactor bump-up.command script and add helix-view-aggregator (#2086)
Qi (Quincy) Qu [Tue, 10 May 2022 20:25:49 +0000 (13:25 -0700)] 
Refactor bump-up.command script and add helix-view-aggregator (#2086)

Refactor bump-up.command script and add helix-view-aggregator

5 months agoFix race condition between instance drop and participant history update (#2073)
Qi (Quincy) Qu [Tue, 3 May 2022 21:38:40 +0000 (14:38 -0700)] 
Fix race condition between instance drop and participant history update (#2073)

Fix race condition between instance drop and participant history update

5 months agoIssue #1970: allow clients to prevent HelixProperty from cloning the ZNRecord (#2072)
Richard Startin [Thu, 28 Apr 2022 23:42:52 +0000 (00:42 +0100)] 
Issue #1970: allow clients to prevent HelixProperty from cloning the ZNRecord (#2072)

5 months agofix issue #2064: bug where RuntimeJobDag.generateJobList could loop until parallelism...
Richard Startin [Thu, 28 Apr 2022 16:50:14 +0000 (17:50 +0100)] 
fix issue #2064: bug where RuntimeJobDag.generateJobList could loop until parallelism is reached when in JobQueue mode (#2065)

5 months agoMaintenanceManagementService improvement - Allow implementation of OperationInterfac...
xyuanlu [Wed, 27 Apr 2022 22:42:51 +0000 (15:42 -0700)] 
MaintenanceManagementService  improvement - Allow implementation of OperationInterface to return null (#2035)

Allow implementation of OperationInterface to return null

5 months agosupport common input for operational input on Maintenance Management API (#2055)
xyuanlu [Wed, 27 Apr 2022 22:42:34 +0000 (15:42 -0700)] 
support common input for operational input on Maintenance Management API  (#2055)

support common input for operational input on Maintenance Management API

5 months agoDo not proceed with cluster creation if addCluster() fails. (#2068)
Komal Desai [Wed, 27 Apr 2022 22:42:08 +0000 (15:42 -0700)] 
Do not proceed with cluster creation if addCluster() fails. (#2068)

In Helix-Tools - ClusterSetup::addCluster() doesn't check return value
of HelixAdmin::addCluster() method. It proceeds even when the call
returns failure.
Check the return status of addCluster() and throw an exception if cluster creation fails.

5 months agoMore term cleanup for tutorial website (#2059)
Qi (Quincy) Qu [Wed, 27 Apr 2022 22:41:06 +0000 (15:41 -0700)] 
More term cleanup for tutorial website (#2059)

5 months agomove to Apache Analytics(Matomo)
Olivier Lamy [Mon, 25 Apr 2022 08:13:54 +0000 (18:13 +1000)] 
move to Apache Analytics(Matomo)

Signed-off-by: Olivier Lamy <olamy@apache.org>
5 months agouse reflow version just released and working with recent version maven-site-plugin
Olivier Lamy [Mon, 25 Apr 2022 08:02:49 +0000 (18:02 +1000)] 
use reflow version just released and working with recent version maven-site-plugin

Signed-off-by: Olivier Lamy <olamy@apache.org>
5 months agofix website deployment
Olivier Lamy [Sat, 23 Apr 2022 04:39:46 +0000 (14:39 +1000)] 
fix website deployment

Signed-off-by: Olivier Lamy <olamy@apache.org>
5 months agoFix release note for Log4j version
Junkai Xue [Fri, 22 Apr 2022 21:35:40 +0000 (14:35 -0700)] 
Fix release note for Log4j version

5 months agoUpdate menu bar
Junkai Xue [Thu, 21 Apr 2022 21:29:45 +0000 (14:29 -0700)] 
Update menu bar

5 months agoAdd bump snapshot script and bump snapshort
Junkai Xue [Thu, 21 Apr 2022 21:19:58 +0000 (14:19 -0700)] 
Add bump snapshot script and bump snapshort

5 months agoupgrade xstream to security issues (#2046)
PJ Fanning [Wed, 20 Apr 2022 21:48:29 +0000 (23:48 +0200)] 
upgrade xstream to security issues (#2046)

5 months agoReplace non-inclusive terms in quickstart (#2050)
Qi (Quincy) Qu [Wed, 20 Apr 2022 20:55:40 +0000 (13:55 -0700)] 
Replace non-inclusive terms in quickstart (#2050)

5 months ago[issue-1728] upgrade guava due to cve (#2042)
PJ Fanning [Tue, 19 Apr 2022 18:21:45 +0000 (20:21 +0200)] 
[issue-1728] upgrade guava due to cve (#2042)

5 months agoupgrade jackson to 2.12.6.1 due to cve (#2043)
PJ Fanning [Tue, 19 Apr 2022 18:21:11 +0000 (20:21 +0200)] 
upgrade jackson to 2.12.6.1 due to cve (#2043)

5 months agoAdd 1.0.3 releasenote
Junkai Xue [Mon, 18 Apr 2022 22:18:22 +0000 (15:18 -0700)] 
Add 1.0.3 releasenote

5 months agoReplace non-inclusive terms in tutorial.md (#2039)
Qi (Quincy) Qu [Mon, 18 Apr 2022 17:57:29 +0000 (10:57 -0700)] 
Replace non-inclusive terms in tutorial.md (#2039)

5 months agoCode refactor and cleanup on instance validation (#2032)
Qi (Quincy) Qu [Mon, 18 Apr 2022 17:57:15 +0000 (10:57 -0700)] 
Code refactor and cleanup on instance validation (#2032)

Unify the usage of checking instance enable/disable using InstanceValidationUtil

5 months agoFixed size history for Scheduled Workflow tasks (#2036)
Komal Desai [Sat, 16 Apr 2022 22:31:19 +0000 (15:31 -0700)] 
Fixed size history for Scheduled Workflow tasks (#2036)

Once we execute scheduled workflow task, we append entry to history.
Each entry is of the format "<taskname>-<timestamp>"

But we never purged old entries.
This will result in hitting size limit of Znode.

Introducing fixed size history of 20 and purge all the previous entries.

5 months agoupgrade snakeyaml to v1.30 due to cve (#2041)
PJ Fanning [Sat, 16 Apr 2022 22:30:31 +0000 (00:30 +0200)] 
upgrade snakeyaml to v1.30 due to cve (#2041)

5 months ago[issue-1727] upgrade to commons-io 2.11.0 due to cve (#2040)
PJ Fanning [Sat, 16 Apr 2022 22:29:52 +0000 (00:29 +0200)] 
[issue-1727] upgrade to commons-io 2.11.0 due to cve (#2040)

5 months agoRemove temp file
Junkai Xue [Fri, 15 Apr 2022 08:10:45 +0000 (01:10 -0700)] 
Remove temp file

5 months agoFix website deployment
Junkai Xue [Fri, 15 Apr 2022 07:24:42 +0000 (00:24 -0700)] 
Fix website deployment

5 months agoRemove archived versions
Junkai Xue [Thu, 14 Apr 2022 19:02:38 +0000 (12:02 -0700)] 
Remove archived versions

5 months agoUpdate dependencies and fix compile errors
Qi (Quincy) Qu [Tue, 5 Apr 2022 23:21:06 +0000 (16:21 -0700)] 
Update dependencies and fix compile errors

Unit test fix and code style improvement.

5 months agoHELIX-1233: fix broken test in helix view aggregator module
Harry Zhang [Fri, 10 Aug 2018 20:26:36 +0000 (13:26 -0700)] 
HELIX-1233: fix broken test in helix view aggregator module

RB=1388194
BUG=HELIX-1233
G=helix-reviewers
R=jxue,lxia,jjwang,hulee
A=lxia

5 months agominor improvements
Harry Zhang [Tue, 8 May 2018 19:27:44 +0000 (12:27 -0700)] 
minor improvements

RB=1304338
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang

5 months agoHELIX-711: implement distributed state model for helix view aggregator
Harry Zhang [Fri, 27 Apr 2018 21:19:19 +0000 (14:19 -0700)] 
HELIX-711: implement distributed state model for helix view aggregator

RB=1295025
BUG=HELIX-711
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=lxia,jjwang

5 months agoHELIX-708: adding basic metrics to HelixViewAggregator
hrzhang [Fri, 2 Mar 2018 02:46:54 +0000 (18:46 -0800)] 
HELIX-708: adding basic metrics to HelixViewAggregator

RB=1237803
BUG=HELIX-708
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang

5 months agoHELIX-797: add mssing package def to helix-view-aggregator's pom and ivy files
hrzhang [Tue, 27 Feb 2018 19:34:25 +0000 (11:34 -0800)] 
HELIX-797: add mssing package def to helix-view-aggregator's pom and ivy files

RB=1233804
BUG=HELIX-797
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang

5 months agoChange RoutingTableProvider to support direct aggregating routing information from...
Lei Xia [Tue, 13 Feb 2018 18:15:55 +0000 (10:15 -0800)] 
Change RoutingTableProvider to support direct aggregating routing information from CurrentStates in each liveinstance. When sourceDataType is set as CurrentState, RoutingTableProvider will listen on CurrentStateChanges and refresh routing table from CurrentStates upon changes.

RB=1221442
G=helix-reviewers
A=jxue

5 months agoHELIX-705: Implement ViewClusterRefresher logic and tests
hrzhang [Tue, 6 Feb 2018 04:30:39 +0000 (20:30 -0800)] 
HELIX-705: Implement ViewClusterRefresher logic and tests

RB=1213810
BUG=HELIX-776
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang,jxue

5 months agoHELIX-705: implemented SourceClusterDataProvider's core logic and related tests
hrzhang [Tue, 6 Feb 2018 19:21:27 +0000 (11:21 -0800)] 
HELIX-705: implemented SourceClusterDataProvider's core logic and related tests

RB=1205694
BUG=HELIX-775
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang

5 months agoHELIX-705: implement view cluster config change related logics and tests
hrzhang [Wed, 24 Jan 2018 01:55:23 +0000 (17:55 -0800)] 
HELIX-705: implement view cluster config change related logics and tests

RB=1205359
BUG=HELIX-705
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=lxia

5 months agoHELIX-705: create interfaces and interactions among HelixViewAggregator components
hrzhang [Wed, 24 Jan 2018 01:55:23 +0000 (17:55 -0800)] 
HELIX-705: create interfaces and interactions among HelixViewAggregator components

RB=1205300
BUG=HELIX-705
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jjwang,jxue

5 months agoHELIX-705: created project infrastructure for helix-view-aggregator module
hrzhang [Wed, 24 Jan 2018 01:02:31 +0000 (17:02 -0800)] 
HELIX-705: created project infrastructure for helix-view-aggregator module

RB=1202163
BUG=HELIX-705
G=helix-reviewers
R=lxia,jjwang,jxue,erkim
A=jxue

5 months agoBump jackson-databind in /metadata-store-directory-common (#2029)
dependabot[bot] [Mon, 11 Apr 2022 17:48:08 +0000 (13:48 -0400)] 
Bump jackson-databind in /metadata-store-directory-common (#2029)

Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.11.0 to 2.12.6.1.
- [Release notes](https://github.com/FasterXML/jackson/releases)
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-databind
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months agoBump moment from 2.22.2 to 2.29.2 in /helix-front (#2025)
dependabot[bot] [Mon, 11 Apr 2022 14:15:51 +0000 (10:15 -0400)] 
Bump moment from 2.22.2 to 2.29.2 in /helix-front (#2025)

Bumps [moment](https://github.com/moment/moment) from 2.22.2 to 2.29.2.
- [Release notes](https://github.com/moment/moment/releases)
- [Changelog](https://github.com/moment/moment/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/moment/moment/compare/2.22.2...2.29.2)

---
updated-dependencies:
- dependency-name: moment
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5 months agoImplement DefaultCloudEventCallbackImpl (#1995)
Molly Gao [Wed, 6 Apr 2022 17:53:45 +0000 (10:53 -0700)] 
Implement DefaultCloudEventCallbackImpl (#1995)

Implement a default callback implementation for Helix cloud event listeners.

5 months agoFix TestCloudEventCallbackProperty by bypassing connecting to zk (#2017)
Molly Gao [Wed, 6 Apr 2022 17:53:22 +0000 (10:53 -0700)] 
Fix TestCloudEventCallbackProperty by bypassing connecting to zk (#2017)

Due to logical change in ZKHelixManager constructor in a #1986, TestCloudEventCallbackProperty broke because in this test it doesn't connect to zookeeper server.
To fix this test, we separated MockCloudEventAwareHelixManager (previously called MockEventAwareZKHelixManager, nested inside TestCloudEventCallbackProperty)into a class, and include all and only the logics related to cloud events in MockCloudEventAwareHelixManager. More detailed, we mock a cloud config object retrieved from zk to bypass connection to zk.

5 months agoEnable HelixManager as an event listener (#1978)
Molly Gao [Mon, 21 Mar 2022 20:31:35 +0000 (13:31 -0700)] 
Enable HelixManager as an event listener (#1978)

Make helix manager cloud event aware by registering a cloud event listener when connect Helix manager

5 months agoAdd event handler and event listener interface (#1976)
Molly Gao [Thu, 10 Mar 2022 22:05:33 +0000 (14:05 -0800)] 
Add event handler and event listener interface (#1976)

This commit creates a skeleton for event handling framework and adds the following classes:
CloudEventListener interface
CloudEventHandler class
CloudEventHandlerFactory class

5 months agoImprove ZkClientMonitor and ZkClientPathMonitor performance (#2021)
Henri Hagberg [Thu, 7 Apr 2022 14:09:49 +0000 (17:09 +0300)] 
Improve ZkClientMonitor and ZkClientPathMonitor performance (#2021)

Previously, regex matches were used, which was inefficient. This commit does this following:
Replace String#matches with more efficient String#contains in ZkClientPathMonitor
Refactor record* methods in ZkClientMonitor to avoid repetition and simplify matching logic

5 months agoadd type and reason to cluster config (#2006)
xyuanlu [Wed, 6 Apr 2022 01:34:49 +0000 (18:34 -0700)] 
add type and reason to cluster config (#2006)

Add type and reason for batch disable/enable instance

5 months agoRefactor config string processing logic into a util class (#2015)
xyuanlu [Tue, 5 Apr 2022 21:01:02 +0000 (14:01 -0700)] 
Refactor config string processing logic into a util class (#2015)

Refactor config string processing logic into a util class

5 months agoFix TestDropResourceMetricsReset (#2011)
Qi (Quincy) Qu [Tue, 5 Apr 2022 17:41:44 +0000 (10:41 -0700)] 
Fix TestDropResourceMetricsReset (#2011)

Add a temp workaround by manually triggering a CurrentStageChange event for ExternalViewStage computation so that resource monitor can be cleaned up in unit test.

5 months agoPopulate helix cloud property using cloud config (#2005)
Molly Gao [Tue, 5 Apr 2022 17:13:01 +0000 (10:13 -0700)] 
Populate helix cloud property using cloud config (#2005)

Currently when instantiating a zk helix manager, we retrieve cloud config from zk and replace entire HelixCloudProperty object, which may cause some fields that user pass in in HelixCloudProperty that are not included in cloud config get missing. This commit changes the logic to only populate fields in HelixCloudProperty with values that are present in cloud config, and leave other fields unchanged.

5 months agoFix web build and deployment Issue (#2014)
Junkai Xue [Mon, 4 Apr 2022 17:17:49 +0000 (10:17 -0700)] 
Fix web build and deployment Issue (#2014)

5 months agofix Broken Logic in perPartitionHealthCheck (#2012)
xyuanlu [Mon, 4 Apr 2022 17:14:03 +0000 (10:14 -0700)] 
fix Broken Logic in perPartitionHealthCheck (#2012)

fix Broken Logic in perPartitionHealthCheck

5 months agofix repoducible builds issue (#2013)
Hervé Boutemy [Sun, 3 Apr 2022 21:18:07 +0000 (23:18 +0200)] 
fix repoducible builds issue (#2013)

6 months agoUpdate 0.9.9 to be replaced by 0.9.10
Junkai Xue [Sat, 2 Apr 2022 23:29:28 +0000 (16:29 -0700)] 
Update 0.9.9 to be replaced by 0.9.10

6 months agoadd new error message for customized partition check host connection error (#1984)
xyuanlu [Thu, 31 Mar 2022 23:19:00 +0000 (16:19 -0700)] 
add new error message for customized partition check host connection error  (#1984)

Add new error message for customized partition check host connection error

6 months agoauto Exit MM for auto EMM test (#2003)
xyuanlu [Thu, 31 Mar 2022 23:18:25 +0000 (16:18 -0700)] 
auto Exit MM for auto EMM test (#2003)

6 months agoAdd instance disable reason (#1993) (#2004)
xyuanlu [Thu, 31 Mar 2022 23:18:11 +0000 (16:18 -0700)] 
Add instance disable reason  (#1993) (#2004)

Add instance disable reason

6 months agoImplement deactivate rest API (#1988)
Qi (Quincy) Qu [Thu, 24 Mar 2022 16:51:46 +0000 (09:51 -0700)] 
Implement deactivate rest API (#1988)

Implement deactivate REST API
Added a new REST API for deactivating cluster from supercluster.

6 months agoRead cloud config from zk and propagate to HelixManagerProperty in ZkHelixManager...
xyuanlu [Thu, 24 Mar 2022 16:30:00 +0000 (09:30 -0700)] 
Read cloud config from zk and propagate to HelixManagerProperty in ZkHelixManager constructor (#1986)

Read cloud config from zk and propagate to HelixManagerProperty in ZkHelixManager constructor.

6 months agoAdd Authorization Components to helix-rest (#1967) (#1981)
Neal Sun [Mon, 14 Mar 2022 23:30:43 +0000 (16:30 -0700)] 
Add Authorization Components to helix-rest (#1967) (#1981)

* Add Authorization Components to helix-rest

* Address some comments

6 months agoUse ZooKeeper 3.5.9 in zookeeper-api instead (#1977)
Hunter Lee [Sat, 12 Mar 2022 23:22:21 +0000 (18:22 -0500)] 
Use ZooKeeper 3.5.9 in zookeeper-api instead (#1977)

With the upgrade of apache zookeeper version, snappy-java was missing in the dependency. This commit adds snappy-java and removes unused imports in osgi declaration. Also, using ZooKeeper 3.5.9 in zookeeper-api instead because 3.6.0+ causes some tests to fail in zookeeper-api.

6 months agoAdd 3 Zookeeper CreateMode types to AccessOption (#1975)
Ramin Bashizade [Wed, 9 Mar 2022 17:29:31 +0000 (09:29 -0800)] 
Add 3 Zookeeper CreateMode types to AccessOption (#1975)

This commit adds the 3 missing CreateMode types to AccessOption
class in helix-core: CONTAINER, PERSISTENT_WITH_TTL, and
PERSISTENT_SEQUENTIAL_WITH_TTL.

6 months agoUpgrading Zookeeper version to 3.6.13 to enable zk client SSL/TLS
rahulrane50 [Tue, 8 Mar 2022 21:42:44 +0000 (13:42 -0800)] 
Upgrading Zookeeper version to 3.6.13 to enable zk client SSL/TLS

Upgrading Zookeeper version to 3.6.13 to enable zk client SSL/TLS support

7 months ago[HELIX-862] s/maintainence/maintenance docs fix (#1968)
Micah Stubbs [Thu, 3 Mar 2022 22:08:34 +0000 (14:08 -0800)] 
[HELIX-862] s/maintainence/maintenance docs fix (#1968)

7 months agoAdd rest endpoint for virtual topology group (#1958)
Qi (Quincy) Qu [Wed, 16 Feb 2022 17:55:56 +0000 (12:55 -0500)] 
Add rest endpoint for virtual topology group (#1958)

7 months agoImplement java API and utils for virtual topology group (#1935)
Qi (Quincy) Qu [Tue, 8 Feb 2022 21:53:40 +0000 (16:53 -0500)] 
Implement java API and utils for virtual topology group (#1935)

Add comment to VirtualTopologyGroupService.

7 months agoIntroduce VirtualTopologyGroup and its assignment logic with benchmark. (#1948)
Qi (Quincy) Qu [Thu, 3 Feb 2022 20:18:46 +0000 (12:18 -0800)] 
Introduce VirtualTopologyGroup and its assignment logic with benchmark. (#1948)

* Cleanup unused assignment schemes and minor change.

* Further refactor and code cleanup.

7 months agoFix #1946 -- Refactor and move ClusterTopologyConfig
Qi (Quincy) Qu [Sat, 29 Jan 2022 00:14:09 +0000 (16:14 -0800)] 
Fix #1946 -- Refactor and move ClusterTopologyConfig

Move ClusterTopologyConfig from nested to a standalone class in helix/model and to be used by virtual topology group logic.

7 months agoUse final remaining capacity when computing weighted score (#1961)
xyuanlu [Wed, 16 Feb 2022 21:52:48 +0000 (13:52 -0800)] 
Use final remaining capacity when computing weighted score (#1961)

WAGED improvement: Use final remaining capacity when computing weighted score

7 months agoRemove WAGED sorting for each assignment (#1959)
xyuanlu [Tue, 15 Feb 2022 00:58:05 +0000 (16:58 -0800)] 
Remove WAGED sorting for each assignment (#1959)

Improve WAGED sorting from n^2 to n*log(n)

7 months agoremove log before write error message to ZNode (#1955)
xyuanlu [Mon, 7 Feb 2022 23:07:16 +0000 (15:07 -0800)] 
remove log before write error message to ZNode (#1955)

Remove message logging before writing ZNode.

7 months agoFixes #1802 - messages intended for instances that are no longer in the cluster ...
Komal Desai [Mon, 7 Feb 2022 23:06:57 +0000 (15:06 -0800)] 
Fixes #1802 - messages intended for instances that are no longer in the cluster (#1951)

In MessageGenerationPhase.java, - process() method populates the list of live instances from cache.

But while generateMessage() method has the sessionIdMap information, it still goes through partition/resource/instance map without checking if instance is still part of the cluster or not.

It is possible that cache has stale entry but that logic needs to be worked separately. But while generating message, we should check if the instance is still there.

So this is a simple change. We need to still look further if cache is getting invalidated properly.

To make sure that the cache properly is handled/refreshed under instance being replaced or deletion - have filled another bug: #1956

7 months agoLet logging framework format exception stack traces (#1954)
Henri Hagberg [Thu, 3 Feb 2022 21:35:05 +0000 (23:35 +0200)] 
Let logging framework format exception stack traces (#1954)

Where possible, logging calls are changed so that logging framework handles exception formatting instead of stack trace being manually formatted using Throwable#getStackTrace

7 months agoAdd new metrics to record ZNRecord compression count. (#1943)
Jiajun Wang [Wed, 2 Feb 2022 20:28:14 +0000 (12:28 -0800)] 
Add new metrics to record ZNRecord compression count. (#1943)

This PR determines if a ZK write request is compressed by calling GZipCompressionUtil. This is an indirect method and can be inaccurate. So the decision is based on trade-offs.

Alternatively, the ZkClientMonitor can be passed into the serializer class and then report compressed write internally. However, this will require multiple changes in the serializer interfaces.
Due to the multiple layers (PathBasedZkSerializer, ZkSerializer) of serializer interfaces definition, it would be very costly to implement the alternative without major refactoring.

8 months agoFix for - Stale message redundant logs
desaikomal [Mon, 31 Jan 2022 02:08:42 +0000 (18:08 -0800)] 
Fix for - Stale message redundant logs

Avoid printing redundant log messages for unrelated partitions and resources.

8 months agoFix Issue#1941 - Incorrect condition caused not to log error message
desaikomal [Sat, 29 Jan 2022 00:16:16 +0000 (16:16 -0800)] 
Fix Issue#1941 - Incorrect condition caused not to log error message

Properly populate the error log messages for partitions and resource names whose replica status is in ERROR state.

8 months agoFix CVE dependency issue (#1927)
CVEDetect [Wed, 19 Jan 2022 15:17:11 +0000 (23:17 +0800)] 
Fix CVE dependency issue (#1927)

8 months agoRemove dependency to an old Jackson v1 library (org.codehaus.jackson:jackson-mapper...
Andrzej Hołowko [Wed, 19 Jan 2022 15:15:31 +0000 (16:15 +0100)] 
Remove dependency to an old Jackson v1 library (org.codehaus.jackson:jackson-mapper-asl) affected by the critical vulnerability: CVE-2019-17267 (#1934)