[NO ISSUE][STO] Delete invalid indexes during cluster global recovery
authorMurtadha Hubail <mhubail@apache.org>
Wed, 31 Mar 2021 18:18:18 +0000 (21:18 +0300)
committerMurtadha Hubail <mhubail@apache.org>
Thu, 1 Apr 2021 10:24:38 +0000 (10:24 +0000)
commit17670aab184fe12fa30dc79376e819e07fac43c4
treedb059ed2b854af2d1af41b5e7c6789514cedcc0d
parentd3827662ad11193341a12d931bf2eb8188f07e59
[NO ISSUE][STO] Delete invalid indexes during cluster global recovery

- user model changes: yes
- storage format changes: no
- interface changes: no

Details:

- Before starting cluster global recovery, send to all NCs valid
  dataset ids from the metadata node.
- Delete any invalid indexes on NCs based on the metadata received
  from the CC.
- Add storage options to enable/disable global storage recovery.
  This allows tests that create storage objects without using the
  metadata node to bypass global cleanup.
- Add storage option to specify the timeout for nodes to perform
  global storage cleanup.
- Add test case for global storage recovery.
- Adapt existing test cases that require bypassing global cleanup.

Change-Id: Idee73e57fa5879c3b9aab5f881bf848e225f874b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/10784
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Murtadha Hubail <mhubail@apache.org>
Reviewed-by: Michael Blow <mblow@apache.org>
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/StorageCleanupRequestMessage.java [new file with mode: 0644]
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/VoidResponse.java [new file with mode: 0644]
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/GlobalRecoveryManager.java
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/GlobalStorageCleanupTest.java [new file with mode: 0644]
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/LSMFlushRecoveryTest.java
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm
asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/StorageProperties.java