Add a metric for task duration in the pending queue (#12492)
authorRocky Chen <40374064+rockc2020@users.noreply.github.com>
Tue, 3 May 2022 03:47:25 +0000 (20:47 -0700)
committerGitHub <noreply@github.com>
Tue, 3 May 2022 03:47:25 +0000 (23:47 -0400)
commit770ad951693f30e6b56af4db430f88f0408d10d5
tree9906a24741b2f2f829f4fbb5bebf9bb8bd1084db
parent785a1eeb9f4e70984b76a4e8900dc5e3e15acc4f
Add a metric for task duration in the pending queue (#12492)

This PR is to measure how long a task stays in the pending queue and emits the value with the metric task/pending/time. The metric is measured in RemoteTaskRunner and HttpRemoteTaskRunner.

An example of the metric:

```
2022-04-26T21:59:09,488 INFO [rtr-pending-tasks-runner-0] org.apache.druid.java.util.emitter.core.LoggingEmitter - {"feed":"metrics","timestamp":"2022-04-26T21:59:09.487Z","service":"druid/coordinator","host":"localhost:8081","version":"2022.02.0-iap-SNAPSHOT","metric":"task/pending/time","value":8,"dataSource":"wikipedia","taskId":"index_parallel_wikipedia_gecpcglg_2022-04-26T21:59:09.432Z","taskType":"index_parallel"}
```

------------------------------------------
Key changed/added classes in this PR

    Emit metric task/pending/time in classes RemoteTaskRunner and HttpRemoteTaskRunner.
    Update related factory classes and tests.
docs/operations/metrics.md
indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunner.java
indexing-service/src/main/java/org/apache/druid/indexing/overlord/RemoteTaskRunnerFactory.java
indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunner.java
indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunnerFactory.java
indexing-service/src/test/java/org/apache/druid/indexing/overlord/RemoteTaskRunnerFactoryTest.java [new file with mode: 0644]
indexing-service/src/test/java/org/apache/druid/indexing/overlord/RemoteTaskRunnerTestUtils.java
indexing-service/src/test/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunnerTest.java