Fix force GC doesn't work under forceAllowCompaction when disk is full (#3205)
authorHang Chen <chenhang@apache.org>
Wed, 20 Apr 2022 13:43:15 +0000 (21:43 +0800)
committerGitHub <noreply@github.com>
Wed, 20 Apr 2022 13:43:15 +0000 (21:43 +0800)
commitf0ff353d02ec56715d671040717870ec5f1a7630
treedd9e95613be46cee3c8ed782a067f375fa074f4b
parent16c9766b952fb3d1fb42d6a308e8525c9ffddbf8
Fix force GC doesn't work under forceAllowCompaction when disk is full (#3205)

## Motivation
When I set `forceAllowCompaction=true` and one ledger disk reaches max usage threshold and transfer bookie to readOnly mode, I expire some pulsar topics or delete some topics to free up disk space. I found that ledger compression cannot be triggered when using `curl -XPUT http://localhost:8000/api/v1/bookie/gc` command.

The root cause is that when one ledger disk reaches max usage threshold, it will suspend minor and major compaction
https://github.com/apache/bookkeeper/blob/f7579fd13d62ce630ea26638e73f5884da505ec8/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java#L1041-L1058

When we use `curl -XPUT http://localhost:8000/api/v1/bookie/gc` command to trigger compaction, it will be filtered by `suspendMajor` and `suspendMinor` flag.

https://github.com/apache/bookkeeper/blob/f7579fd13d62ce630ea26638e73f5884da505ec8/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java#L416-L444

It will lead to
- The bookie won't clean up deleted ledgers
- Ledger disk can't free up disk usage
- Bookie can't recover from readOnly state into Writeable state.

And then we can only trigger compaction by the following steps.
- Increase max disk usage threshold
- Restart the bookie
- Use command `curl -XPUT http://localhost:8000/api/v1/bookie/gc` to trigger compaction

### Changes
1. Don't take the `suspendMajor` and `suspendMinor` flag into consideration when setting `forceAllowCompaction=true` and triggered by force GC.
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/DbLedgerStorage.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/SingleDirectoryDbLedgerStorage.java
bookkeeper-server/src/main/java/org/apache/bookkeeper/server/http/service/TriggerGCService.java
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/MockLedgerStorage.java