DRILL-6494: Drill Plugins Handler
authorVitalii Diravka <vitalii.diravka@gmail.com>
Tue, 3 Jul 2018 17:23:03 +0000 (20:23 +0300)
committerGitHub <noreply@github.com>
Tue, 3 Jul 2018 17:23:03 +0000 (20:23 +0300)
commit0ae7035de390c699f574d1ec25d45cd8b20a8b94
tree3012eafa62d188030069d235649bce2c7282c674
parent069c3049f1a500e5ae0b47caeebc5856ab182b73
DRILL-6494: Drill Plugins Handler

- Storage Plugins Handler service is used op the Drill start-up stage and it updates storage plugins configs from
  storage-plugins-override.conf file. If plugins configs are present in the persistence store - they are updated,
  otherwise bootstrap plugins are updated and the result configs are loaded to persistence store. If the enabled
  status is absent in the storage-plugins-override.conf file, the last plugin config enabled status persists.
- 'drill.exec.storage.action_on_plugins_override_file' Boot option is added. This is the action, which should be
  performed on the storage-plugins-override.conf file after successful updating storage plugins configs.
  Possible values are: "none" (default), "rename" and "remove".
- The "NULL" issue with updating Hive plugin config by REST is solved. But clients are still being instantiated for disabled
  plugins - DRILL-6412.
- "org.honton.chas.hocon:jackson-dataformat-hocon" library is added for the proper deserializing HOCON conf file
- additional refactoring: "com.typesafe:config" and "org.apache.commons:commons-lang3" are placed into DependencyManagement
  block with proper versions; correct properties for metrics in "drill-override-example.conf" are specified

closes #1345
35 files changed:
common/pom.xml
common/src/main/java/org/apache/drill/common/config/CommonConstants.java
common/src/main/java/org/apache/drill/common/config/DrillConfig.java
common/src/main/java/org/apache/drill/common/scanner/ClassPathScanner.java
common/src/main/java/org/apache/drill/exec/util/ActionOnFile.java [new file with mode: 0644]
contrib/storage-hbase/src/main/resources/bootstrap-storage-plugins.json
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/schema/HiveSchemaFactory.java
contrib/storage-hive/core/src/main/resources/bootstrap-storage-plugins.json
contrib/storage-jdbc/src/test/resources/bootstrap-storage-plugins.json
contrib/storage-kafka/src/main/resources/bootstrap-storage-plugins.json
contrib/storage-mongo/src/main/resources/bootstrap-storage-plugins.json
distribution/src/assemble/bin.xml
distribution/src/resources/drill-override-example.conf
distribution/src/resources/storage-plugins-override-example.conf [new file with mode: 0644]
drill-yarn/pom.xml
exec/java-exec/pom.xml
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/StoragePlugins.java
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/StorageResources.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/NamedStoragePluginConfig.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginMap.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistryImpl.java
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginsHandler.java [new file with mode: 0644]
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginsHandlerService.java [new file with mode: 0644]
exec/java-exec/src/main/java/org/apache/drill/exec/util/StoragePluginTestUtils.java
exec/java-exec/src/main/resources/bootstrap-storage-plugins.json
exec/java-exec/src/main/resources/drill-module.conf
exec/java-exec/src/test/java/org/apache/drill/exec/impersonation/BaseTestImpersonation.java
exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestCTTAS.java
exec/java-exec/src/test/java/org/apache/drill/test/ClusterFixture.java
exec/java-exec/src/test/java/org/apache/drill/test/ClusterMockStorageFixture.java
logical/pom.xml
logical/src/main/java/org/apache/drill/common/config/LogicalPlanPersistence.java
logical/src/main/java/org/apache/drill/common/logical/StoragePluginConfig.java
pom.xml