bookkeeper.git
2 hours agoBump vertx version from 3.9.8 to 4.3.2 (#3435) master
ZhangJian He [Sun, 7 Aug 2022 14:09:36 +0000 (22:09 +0800)] 
Bump vertx version from 3.9.8 to 4.3.2 (#3435)

5 days agoRevert some unwanted changes in commit 3309 (#3438)
Yong Zhang [Tue, 2 Aug 2022 05:36:35 +0000 (13:36 +0800)] 
Revert some unwanted changes in commit 3309 (#3438)

---

*Motivation*

There are some unwanted changes committed by 3309
https://github.com/apache/bookkeeper/pull/3309

6 days agoBump terser from 5.12.0 to 5.14.2 in /site3/website (#3414)
dependabot[bot] [Mon, 1 Aug 2022 13:54:46 +0000 (21:54 +0800)] 
Bump terser from 5.12.0 to 5.14.2 in /site3/website (#3414)

Bumps [terser](https://github.com/terser/terser) from 5.12.0 to 5.14.2.
- [Release notes](https://github.com/terser/terser/releases)
- [Changelog](https://github.com/terser/terser/blob/master/CHANGELOG.md)
- [Commits](https://github.com/terser/terser/commits)

---
updated-dependencies:
- dependency-name: terser
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6 days agoBump version for release 4.16 (#3436)
Hang Chen [Mon, 1 Aug 2022 13:53:56 +0000 (21:53 +0800)] 
Bump version for release 4.16 (#3436)

### Descriptions of the changes in this PR:

Updated py client's version to 4.16

### Motivation

Preparing for the release
https://bookkeeper.apache.org/community/release_guide/#change-python-client-version

6 days agoIssue #2908: Replace unsafe NoEntryException with IOException (#2909)
Jack Vanlightly [Mon, 1 Aug 2022 07:11:35 +0000 (09:11 +0200)] 
Issue #2908: Replace unsafe NoEntryException with IOException (#2909)

* Replace unsafe NoEntryException with IOException

Throwing a NoEntryException from the entry logger
for an entry that the index says should exist is
unsafe. It can cause ledger truncation during ledger
recovery. It only takes a single false NoSuchEntry
response to trigger truncation.

NoEntryException should only be thrown from inside
ledger storage if the entry is not found in the index.

* fix CI

Co-authored-by: chenhang <chenhang@apache.org>
7 days agoBP-41 Add flag to enable/disable BookieAddressResolver (#3356)
Masahiro Sakamoto [Sun, 31 Jul 2022 07:12:06 +0000 (16:12 +0900)] 
BP-41 Add flag to enable/disable BookieAddressResolver (#3356)

### Motivation

With BP-41, the BookKeeper client now needs a request to ZooKeeper to resolve the address from each bookie ID.

In the following document, there is a description of the flag ~`enableBookieAddressResolver`~ `bookieAddressResolverEnabled` for disabling this feature by regarding the bookie ID as an address or hostname, but it seems that this has not been implemented yet.
https://github.com/apache/bookkeeper/blob/master/site3/website/src/pages/bps/BP-41-bookieid.md

I implemented this because I want this flag to reduce the number of requests to ZK.

### Changes

Added a flag named ~`enableBookieAddressResolver`~ `bookieAddressResolverEnabled` to the client configuration. If this flag is false, use `BookieAddressResolverDisabled` instead of `DefaultBookieAddressResolver` as the address resolver for bookies. `BookieAddressResolverDisabled` regards a bookie ID to be in legacy format, i.e. "address:port" or "hostname:port", and returns the address of that bookie without access to ZK.

Master Issue: #2396

7 days agoindexDir compatible fix for issue #3430 (#3433)
StevenLuMT [Sun, 31 Jul 2022 04:07:21 +0000 (12:07 +0800)] 
indexDir compatible fix for issue #3430 (#3433)

Descriptions of the changes in this PR:

### Motivation
Fixes https://github.com/apache/bookkeeper/issues/3430, make cookie validation is compatible with old version.
I think we should check indexDirs when configuration‘s indexDirectories is setted  rather than avoiding the check

### Changes
1. add compatible check
2. add testcase to cover the new code

7 days agoissue #2879 : let bookie quit if journal thread exit (#2887)
AloysZhang [Sun, 31 Jul 2022 03:30:45 +0000 (11:30 +0800)] 
issue #2879 : let bookie quit if journal thread exit (#2887)

Descriptions of the changes in this PR:
fix #2879
This pull request let bookie quit when there's journal thread exit

### Motivation

As described in #2879, now if a bookie has multi journal directories means it has multi journal thread. Once a journal thread exits, the bookie will be unhealthy due to the block of all bookie-io threads, and then the bookie will not work but progress is still alive.
This pull request tries to fix this problem.

### Changes

check the journal thread alive in a fixed interval, let bookie quit once there's a journal thread exit

8 days agoupgrade log4j2 to 2.18.0 (#3434)
Hang Chen [Sat, 30 Jul 2022 15:38:29 +0000 (23:38 +0800)] 
upgrade log4j2 to 2.18.0 (#3434)

### Modification
Upgrade log4j2 from 2.17.2 to 2.18.0

9 days agoBump puppycrawl checkstyle version from 7.8.2 to 9.3 (#3311)
ZhangJian He [Fri, 29 Jul 2022 13:12:53 +0000 (21:12 +0800)] 
Bump puppycrawl checkstyle version from 7.8.2 to 9.3 (#3311)

### Changes
- Bump puppycrawl checkstyle version from 6.19 to 10.3
- adapt the `static import, SPACE LINE, import` order

### why so many files are changed
Former checkstyle config is not strict, we have different importOrder styles in code. Most of it (2100+) is `static import, SPACE LINE, import`. Others(366) only needs to remove or add space line.

9 days ago[conf] minorCompactionInterval should be greater than gcWaitTime (#2116)
Penghui Li [Fri, 29 Jul 2022 12:11:21 +0000 (20:11 +0800)] 
[conf] minorCompactionInterval should be greater than gcWaitTime (#2116)

9 days ago[ISSUE 2637] Fix jvm_memory_direct_bytes_used metrics when using jdk11+ (#3252)
ZhangJian He [Fri, 29 Jul 2022 10:38:27 +0000 (18:38 +0800)] 
[ISSUE 2637] Fix jvm_memory_direct_bytes_used metrics when using jdk11+ (#3252)

Fix #2637 #3247

### Motivation
The mertics about `jvm_memory_direct_bytes_used` is acquired by netty's `PlatformDependent#DIRECT_MEMORY_COUNTER`. Which can only acquired the memory used by netty.

### Changes
- use `java.nio.Bits#RESERVED_MEMORY` for jvm direct memory metrics.
- add tests to ensure `jvm_memory_direct_bytes_max` and `jvm_memory_direct_bytes_used` gets value.

9 days agoadd metric cbThreadPoolQueueSize (#3424)
LinChen [Fri, 29 Jul 2022 07:30:20 +0000 (15:30 +0800)] 
add metric cbThreadPoolQueueSize (#3424)

### Motivation
Add a new metric cbThreadPoolQueueSize to count the queue size in the thread pool cbThreadPool.

9 days agofix Flaky-test: testBookieShouldTurnWritableFromReadOnly (#3423)
wenbingshen [Fri, 29 Jul 2022 07:22:59 +0000 (15:22 +0800)] 
fix Flaky-test: testBookieShouldTurnWritableFromReadOnly (#3423)

### Motivation
Fixes issue #3422

1.`waitForReadOnlyBookie` adds another `listener` thread to observe the node status of bookie, which may be out of sync with the triggering of node changes(`onClusterChanged`) in `EnsemblePlacementPolicy`.This `sequence` leads to flaky test.

2.Expose bookie nodes in EnsemblePlacementPolicy to client.

3.Just change from watching zk to thread sleep.

We currently don't have any additional requirements for getting bookie nodes on the client side, so it's better to just use thread sleep here.

### Changes
Change from `waitForReadOnlyBookie` and `waitForWritableBookie` to thread `sleep`.
3 seconds is enough for most.

9 days agoMake sure the LedgerHandle close callback can be completed when encounter exception...
Penghui Li [Fri, 29 Jul 2022 06:56:48 +0000 (14:56 +0800)] 
Make sure the LedgerHandle close callback can be completed when encounter exception (#2913)

* Make sure the LedgerHandle close callback can be completed when encounter exception.

9 days agoBump puppycrawl checkstyle version from 6.19 to 7.8.2 (#3425)
ZhangJian He [Fri, 29 Jul 2022 01:30:47 +0000 (09:30 +0800)] 
Bump puppycrawl checkstyle version from 6.19 to 7.8.2 (#3425)

### Changes
Bump puppycrawl checkstyle version to 7.8.2. Fix violation

10 days agoFix the infinite waiting for shutdown due to throttler limit (#2942)
wenbingshen [Thu, 28 Jul 2022 03:41:52 +0000 (11:41 +0800)] 
Fix the infinite waiting for shutdown due to throttler limit (#2942)

Descriptions of the changes in this PR:
### Motivation

If the compactor is limited, the shutdown priority should be higher than waiting for RateLimiter.acquire.

### Changes

According to @hangc0276 suggestion, when processing the shutdown logic of `GarbageCollectorThread`, we should check the status of the `newScanner.process` method. If the status is false, throw an `IOException` and stop compact immediately.

Master Issue: #2941

10 days agoFix wrong ledger id parse radix for index relocation file in IndexPersistenceMgr...
Kezhu Wang [Thu, 28 Jul 2022 02:07:03 +0000 (10:07 +0800)] 
Fix wrong ledger id parse radix for index relocation file in IndexPersistenceMgr (#2944)

Descriptions of the changes in this PR:

### Motivation
1. `IndexPersistenceMgr` fails to start after crashing index file movement.

### Changes
1. Add test to assert index file relocation
2. Fix wrong ledger id parse radix for index relocation file in IndexPersistenceMgr

Master Issue: None

10 days agofix check style (#3429)
Hang Chen [Thu, 28 Jul 2022 01:33:35 +0000 (09:33 +0800)] 
fix check style (#3429)

### Modification
The CI failed witch check style, fix it.
https://github.com/apache/bookkeeper/runs/7544140069?check_suite_focus=true

11 days agoupdate ALLOCATOR_POOLING_CONCURRENCY default value (#3001)
ssj [Wed, 27 Jul 2022 15:57:48 +0000 (23:57 +0800)] 
update ALLOCATOR_POOLING_CONCURRENCY default value (#3001)

Set a more reasonable default value to avoid OutOfDirectMemoryError.

Descriptions of the changes in this PR:

### Motivation
The default value of  allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator.

Assume:
40 processors
80 arena (processors*2)
2 chunk per arena
16MiB per chunk
JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured.

OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits.

### Changes
Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency.

11 days ago[improve][client] release the bookie from QuarantinedBookies when health check is...
lixinyang [Wed, 27 Jul 2022 15:52:10 +0000 (23:52 +0800)] 
[improve][client] release the bookie from QuarantinedBookies when health check is disabled (#3349)

### Motivation
we need release all quarantined bookies when we use the dynamic config to disable the health check. In some case, we disable the health check when the client has add all bookies to `quarantinedBookies`, then we want to recovery all client's read/write capability immediately.
But now we need wait `BOOKIE_QUARANTINE_TIME_SECONDS` reached or we need restart all client, it's too slowly to recovery the client read/write for us.

### Changes
Release all `QuarantinedBookies`,  after we dynamic disable the  health check.

11 days agoPre break loop when self create layoutZNode succeed. (#3335)
Yan Zhao [Wed, 27 Jul 2022 11:09:07 +0000 (19:09 +0800)] 
Pre break loop when self create layoutZNode succeed. (#3335)

Pre break loop when self create layoutZNode succeed.

11 days agoBP-50: Add reuse journal file feature to support Intel PMem disk (#3194)
Hang Chen [Wed, 27 Jul 2022 11:04:21 +0000 (19:04 +0800)] 
BP-50: Add reuse journal file feature to support Intel PMem disk (#3194)

### Motivation
The implementation of #3193

11 days agoFix typo in bookkeeper-proposals.md (#3427)
ZhangJian He [Wed, 27 Jul 2022 07:03:52 +0000 (15:03 +0800)] 
Fix typo in bookkeeper-proposals.md (#3427)

12 days agoFix the 3141 revert issue (#3283)
xiaolong ran [Tue, 26 Jul 2022 15:22:07 +0000 (23:22 +0800)] 
Fix the 3141 revert issue (#3283)

Signed-off-by: xiaolongran <xiaolongran@tencent.com>
### Motivation

In #3144, we reverted the changes of #2686, but after the revert, the self-increment behavior of deletedEntries was also removed, resulting in deletedEntries No assignment, always 0.

In #2686

<img width="1501" alt="image" src="https://user-images.githubusercontent.com/20965307/169231903-1a0bee03-f602-4c61-9c98-6b832f75648f.png">

In #3144

<img width="1352" alt="image" src="https://user-images.githubusercontent.com/20965307/169232028-658a1182-d8c5-4cfa-8f39-2ed7416ee508.png">

### Changes

- Add `++deletedEntries` for removeOffsetFromDeletedLedgers.

12 days agoRemove useless and duplicated ledger directory usage metric (#3156)
萧易客 [Tue, 26 Jul 2022 13:38:27 +0000 (21:38 +0800)] 
Remove useless and duplicated ledger directory usage metric (#3156)

### Motivation

The disk usage metric and the number of writable directories about ledger directories is duplicated cause `LedgerDirsManager` was created with same ledgerDir more than once, here it is:
1. with `bookie_ledger` scope
https://github.com/apache/bookkeeper/blob/38dc3281f5f175a7d194d4c0d4b255472886187b/bookkeeper-server/src/main/java/org/apache/bookkeeper/server/Main.java#L365-L366
- `bookie_ledger_dir_<ledger directory>_usage`
- `bookie_ledger_writable_dirs`

2. with `bookie` score
https://github.com/apache/bookkeeper/blob/38dc3281f5f175a7d194d4c0d4b255472886187b/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/DbLedgerStorage.java#L155
- `bookie_dir_<ledger directory>_usage`
- `bookie_writable_dirs`

The second one wasn't bind with a `DiskChecker`, so it's metric never changes.

### Changes

Remove second one, remove metrics below:
- `bookie_dir_<ledger directory>_usage`
- `bookie_writable_dirs`

12 days agoOptimize concurrent collection's shrink logic (#3417)
wenbingshen [Tue, 26 Jul 2022 12:39:50 +0000 (20:39 +0800)] 
Optimize concurrent collection's shrink logic (#3417)

### Motivation

Optimize concurrent collection's shrink and clear logic

### Changes
1. Reduce the repeated `Arrays.fill` in the clear process
2. When `capacity` is already equal to `initCapacity`,`rehash` should not be executed
3. Reduce the `rehash` logic in the `clear` process
4. Shrinking must at least ensure `initCapacity`, so as to avoid frequent shrinking and expansion near `initCapacity`, frequent shrinking and expansion, additionally opened `arrays` will consume more memory and affect GC.

If this PR is accepted, I will optimize the same `concurrent collection's shrink and clear logic ` defined in pulsar.

Related to #3061 and #3074

12 days agoConsolidate Netty channel flushes to mitigate syscall overhead (#3383)
Matteo Merli [Tue, 26 Jul 2022 09:34:01 +0000 (02:34 -0700)] 
Consolidate Netty channel flushes to mitigate syscall overhead (#3383)

12 days agoFix autoRecover memory leak. (#3361)
Yan Zhao [Tue, 26 Jul 2022 09:32:40 +0000 (17:32 +0800)] 
Fix autoRecover memory leak. (#3361)

12 days agovalidate diskUsageThreshold and diskUsageLwmThreshold (#3285)
wenbingshen [Tue, 26 Jul 2022 09:24:18 +0000 (17:24 +0800)] 
validate diskUsageThreshold and diskUsageLwmThreshold (#3285)

### Motivation

When `diskUsageThreshold < diskUsageLwmThreshold`, the bookie can be started normally. When the disk usage reaches `diskUsageThreshold` , bookie will automatically switch to `ReadOnly` mode. The `LedgerDirsMonitor` then switches the bookie back to  `read-write` mode since the disk usage is less than `diskUsageLwmThreshold`, the bookie will switch state back and forth frequently.

### Changes
When creating `LedgerDirsMonitor`, we need to validate `diskUsageThreshold` and `diskUsageLwmThreshold` first.

12 days agoavoid init WriteSet when waitForWriteSetMs < 0. (#3325)
Yan Zhao [Tue, 26 Jul 2022 09:18:16 +0000 (17:18 +0800)] 
avoid init WriteSet when waitForWriteSetMs < 0. (#3325)

### Motivation
Avoid init WriteSet when waitForWriteSetMs < 0.
And LedgerHandleAdv didn't recycle WriteSet.

12 days agofix underReplicatedLedgerTotalSize calculate problem. (#3337)
Yan Zhao [Tue, 26 Jul 2022 05:28:34 +0000 (13:28 +0800)] 
fix underReplicatedLedgerTotalSize calculate problem. (#3337)

Descriptions of the changes in this PR:
```
  LongAdder underReplicatedSize = new LongAdder();
        FutureUtils.processList(
                Lists.newArrayList(ledgers),
                ledgerId ->
                    ledgerManager.readLedgerMetadata(ledgerId).whenComplete((metadata, exception) -> {
                        if (exception == null) {
                            underReplicatedSize.add(metadata.getValue().getLength());
                        }
                    }), null);
        underReplicatedLedgerTotalSize.registerSuccessfulValue(underReplicatedSize.longValue());
```
`FutureUtils.processList`  is async process, should record `underReplicatedLedgerTotalSize` when it completed.

12 days agoStable testBookieContinueWritingIfMultipleLedgersPresent test (#3421)
wenbingshen [Tue, 26 Jul 2022 03:34:27 +0000 (11:34 +0800)] 
Stable testBookieContinueWritingIfMultipleLedgersPresent test (#3421)

### Motivation

Fixes issue #2665

When the disk is full, `InterleavedLedgerStorage` creates a new entryLog and index file, copies and moves the old index (newFile) `FileInfo` to the new directory, and finally deletes the old index file.

The old file of `FileInfo` was `deleted`, but it was copied and moved a newFile to the new directory. Therefore, the delete status flag should be restored. If it is not set to false,`checkOpen` will throw `FileInfoDeletedException` when judging the fence status of `ledgerHandle` when adding an entry, causing the writing to fail. When the client changes the `ensemble`, it throws an exception because the test has only two bookie nodes and cannot satisfy the two writeSet: `Not enough non-faulty bookies available`.
```java
    public synchronized boolean isFenced() throws IOException {
        checkOpen(false);
        return (stateBits & STATE_FENCED_BIT) == STATE_FENCED_BIT;
    }

    private synchronized void checkOpen(boolean create, boolean openBeforeClose)
            throws IOException {
        if (deleted) {
            throw new FileInfoDeletedException();
        }
      .....
    }
```

### Changes
When `FileInfo` moves to a new directory by `moveToNewLocation`,delete the old index file and restore the `delete` flag, because the new index file is ready.

Set the disk check interval to the `Integer.MAX_VALUE`, which is stable
`assertEquals("writable dirs should have one dir", 1, ledgerDirsManager
                .getWritableLedgerDirs().size());`
Because once the test thrashes, the disk check thread will restore the full disk to a writable state.
```java
newConf.setDiskCheckInterval(Integer.MAX_VALUE);
```

12 days agoenhance future sync wait. (#3336)
Yan Zhao [Tue, 26 Jul 2022 01:38:35 +0000 (09:38 +0800)] 
enhance future sync wait. (#3336)

Descriptions of the changes in this PR:

Enhance future sync wait.
The same logicment:
https://github.com/apache/bookkeeper/blob/f887f8d7a507800b71b4143a40b0e45902f5f170/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/SyncCallbackUtils.java#L45-L71

12 days agoIf ensembleList is empty, return PlacementPolicyAdherence.FAIL. (#3369)
Yan Zhao [Tue, 26 Jul 2022 01:17:26 +0000 (09:17 +0800)] 
If ensembleList is empty, return PlacementPolicyAdherence.FAIL. (#3369)

Descriptions of the changes in this PR:
If the ensembleList is empty, should not return PlacementPolicyAdherence.MEETS_STRICT.
Return PlacementPolicyAdherence.FAIL to remind user.

2 weeks ago[Issue 3389] Prioritize compaction of entry logs with the lowest amount of remaining...
Andrey Yegorov [Fri, 22 Jul 2022 23:51:01 +0000 (16:51 -0700)] 
[Issue 3389] Prioritize compaction of entry logs with the lowest amount of remaining usable data (#3390)

Descriptions of the changes in this PR:

### Motivation

Prioritize compaction to free up more space faster.

### Changes

doCompactEntryLogs() iterates over entry logs in whatever natural order they happen to be, picks the first with usage below thresholds and starts compacting.

Added a Priority Queue of entry logs to pick ones with the most compactable space first; it also helps when the time for compaction is limited (via majorCompactionMaxTimeMillis / minorCompactionMaxTimeMillis), instead of spending time on rewriting files with more data we'll pick the files with the least amount of data first.

Master Issue: #3389

2 weeks agouse LinkedList to take place of ArrayList. (#3330)
Yan Zhao [Fri, 22 Jul 2022 17:01:40 +0000 (01:01 +0800)] 
use LinkedList to take place of ArrayList. (#3330)

2 weeks agoAdded flag to control whether to transition to read-only mode when any disks full...
Hang Chen [Fri, 22 Jul 2022 16:59:42 +0000 (00:59 +0800)] 
Added flag to control whether to transition to read-only mode when any disks full (#3212)

* Added flag to control whether to transition to read-only mode when any disk is full

* add unit test

* address comments

* add docs and config in conf/bk_server.conf

2 weeks agoSwitch back ordered executor to LinkedBlockingQueue (#3384)
Matteo Merli [Fri, 22 Jul 2022 16:50:56 +0000 (09:50 -0700)] 
Switch back ordered executor to LinkedBlockingQueue (#3384)

2 weeks agoIssue 3206: Flaky-test: BookieZKExpireTest.testBookieServerZKExpireBehaviour (#3415)
Andrey Yegorov [Thu, 21 Jul 2022 09:35:30 +0000 (02:35 -0700)] 
Issue 3206: Flaky-test: BookieZKExpireTest.testBookieServerZKExpireBehaviour (#3415)

* Issue 3206: Flaky-test: BookieZKExpireTest.testBookieServerZKExpireBehaviour

* clear property at the end of the test

2 weeks ago[website] Remove alert for non mantained releases (#3411)
Nicolò Boschi [Tue, 19 Jul 2022 21:58:56 +0000 (23:58 +0200)] 
[website] Remove alert for non mantained releases (#3411)

2 weeks agostage 10: bugfix indexDirs for ConvertToInterleavedStorageCommand (#3409)
StevenLuMT [Tue, 19 Jul 2022 07:11:14 +0000 (15:11 +0800)] 
stage 10: bugfix indexDirs for ConvertToInterleavedStorageCommand (#3409)

Co-authored-by: lushiji <lushiji@didiglobal.com>
2 weeks agoFix JVM exited when running localbookie with jdk17 (#3334)
weiping-code [Mon, 18 Jul 2022 08:22:25 +0000 (16:22 +0800)] 
Fix JVM exited when running localbookie with jdk17 (#3334)

Co-authored-by: ZhangJian He <shoothzj@gmail.com>
3 weeks agoRename bookkeeper metrics to specify OpenMetrics (#3406)
ZhangJian He [Sat, 16 Jul 2022 01:23:34 +0000 (09:23 +0800)] 
Rename bookkeeper metrics to specify OpenMetrics (#3406)

### Motivation
See https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md
A COUNTER needs `metrics_name_total` or `metrics_name_created`

This PR contains metric name broken changes.

### Modifications
Rename counter's `_count` to `_total`

### metrics name broken changes
- rename `FAILED_TO_RESOLVE_NETWORK_LOCATION_COUNTER` to `FAILED_TO_RESOLVE_NETWORK_LOCATION_TOTAL`
- rename `ENSEMBLE_NOT_ADHERING_TO_PLACEMENT_POLICY_COUNT` to `ENSEMBLE_NOT_ADHERING_TO_PLACEMENT_POLICY_TOTAL`
- rename `ACTIVE_ENTRY_LOG_COUNT` to `ACTIVE_ENTRY_LOG_TOTAL`
- rename `MAJOR_COMPACTION_COUNT` to `MAJOR_COMPACTION_TOTAL`
- rename `MINOR_COMPACTION_COUNT` to `MINOR_COMPACTION_TOTAL`
- rename `ACTIVE_LEDGER_COUNT` to `ACTIVE_LEDGER_TOTAL`
- rename `DELETED_LEDGER_COUNT` to `DELETED_LEDGER_TOTAL`
- rename `JOURNAL_FORCE_WRITE_GROUPING_COUNT` to `JOURNAL_FORCE_WRITE_GROUPING_TOTAL`

### Also see
- https://github.com/apache/pulsar/pull/13785
- https://github.com/apache/pulsar/pull/16591
- https://github.com/apache/pulsar/pull/16610
- https://github.com/apache/pulsar/pull/16611

3 weeks agostage 9: bugfix indexDirs for LedgersIndexRebuildOp (#3403)
StevenLuMT [Fri, 15 Jul 2022 07:17:38 +0000 (15:17 +0800)] 
stage 9: bugfix indexDirs for LedgersIndexRebuildOp (#3403)

Co-authored-by: lushiji <lushiji@didiglobal.com>
3 weeks agostage 8: bugfix indexDirs for LedgersIndexCheckOp (#3402)
StevenLuMT [Fri, 15 Jul 2022 07:17:11 +0000 (15:17 +0800)] 
stage 8: bugfix indexDirs for LedgersIndexCheckOp (#3402)

Co-authored-by: lushiji <lushiji@didiglobal.com>
3 weeks agostage 7: just add log for indexDirs (#3401)
StevenLuMT [Fri, 15 Jul 2022 07:16:57 +0000 (15:16 +0800)] 
stage 7: just add log for indexDirs (#3401)

Co-authored-by: lushiji <lushiji@didiglobal.com>
3 weeks agoadd writeThreadQueuedLatency (#3363)
LinChen [Thu, 14 Jul 2022 15:01:35 +0000 (23:01 +0800)] 
add writeThreadQueuedLatency (#3363)

### Motivation
add writeThreadQueueStats :  calculate the time the request stays in the write threadpool queue

3 weeks ago[security] Upgrade Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047 (#3404)
Nicolò Boschi [Thu, 14 Jul 2022 06:10:00 +0000 (08:10 +0200)] 
[security] Upgrade Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047 (#3404)

3 weeks agoMake BookieFileChannel interface public (#3396)
Hang Chen [Wed, 13 Jul 2022 14:39:13 +0000 (22:39 +0800)] 
Make BookieFileChannel interface public (#3396)

### Motivation
The `BookieFileChannel` is package public, and can't be implemented out of the `org.apache.bookkeeper.bookie` package.
We should make it public

### Modification
Make `BookieFileChannel` and `DefaultFileChannel` public

3 weeks agoAdd static declaration for useFallocate in DirectWriter (#3388)
wenbingshen [Wed, 13 Jul 2022 14:37:27 +0000 (22:37 +0800)] 
Add static declaration for useFallocate in DirectWriter (#3388)

### Motivation

The work of the `BP-47` has been very meaningful. I am reading and learning it.

As the title, `useFallocate` in `DirectWriter` has no practical meaning, because the judgment of SystemUtils.IS_OS_LINUX always happens.

3 weeks agoUpgrading JNA from 3.2.7 to 5.12.1 (#3400)
Andrey Yegorov [Wed, 13 Jul 2022 02:46:48 +0000 (19:46 -0700)] 
Upgrading JNA from 3.2.7 to 5.12.1 (#3400)

Descriptions of the changes in this PR:

### Motivation

Upgrading JNA

### Changes

changed dependency version

Master Issue: #3394

3 weeks agostage 5: bugfix indexDirs for readLedgerIndexEntries (#3398)
StevenLuMT [Wed, 13 Jul 2022 02:45:47 +0000 (10:45 +0800)] 
stage 5: bugfix indexDirs for readLedgerIndexEntries (#3398)

Descriptions of the changes in this PR:

### Motivation

planning for index dir: [mail talking](https://lists.apache.org/thread/r657jf55khl59bbqltj2s95107lbkr0w)
stage 5 :
1)  fix indexDirs for EntryLocationIndex's tool

### Changes

1.  bugfix indexDirs for EntryLocationIndex's tool
2. add index's testcase for testReadLedgerIndexEntries
3. bugfix findexDirs for TestBase

3 weeks agoShut down ReplicationWorker and Auditor on non-recoverable ZK error (#3374)
Andrey Yegorov [Tue, 12 Jul 2022 14:57:19 +0000 (07:57 -0700)] 
Shut down ReplicationWorker and Auditor on non-recoverable ZK error (#3374)

3 weeks agostage 6: bugfix indexDirs for LocationsIndexRebuildOp (#3399)
StevenLuMT [Tue, 12 Jul 2022 07:47:48 +0000 (15:47 +0800)] 
stage 6: bugfix indexDirs for LocationsIndexRebuildOp (#3399)

Co-authored-by: lushiji <lushiji@didiglobal.com>
3 weeks agoclean code: remove unused JOURNAL_NUM_FORCE_WRITES (#3379)
LinChen [Tue, 12 Jul 2022 00:12:46 +0000 (08:12 +0800)] 
clean code: remove unused JOURNAL_NUM_FORCE_WRITES (#3379)

### Motivation
clean code: remove unused JOURNAL_NUM_FORCE_WRITES

3 weeks agostage 2.3 ~ 2.4: (#3397)
StevenLuMT [Mon, 11 Jul 2022 22:58:38 +0000 (06:58 +0800)] 
stage 2.3 ~ 2.4: (#3397)

Descriptions of the changes in this PR:

### Motivation

planning for index dir: [mail talking](https://lists.apache.org/thread/r657jf55khl59bbqltj2s95107lbkr0w)
stage 3~4 :
1) disk monitor(DiskChecker/LedgerDirsMonitor) have checked indexDir,it's good;
2) add indexDirs in statsLogger, and add indexDirs's Listener.

### Changes

1. add indexDirs in statsLogger, and add indexDirs's Listener.
2. update index's testcase for SingleDirectoryDbLedgerStorage

4 weeks agoRemove gradle usage from Javadoc gen script (#3386)
Nicolò Boschi [Thu, 7 Jul 2022 23:54:28 +0000 (01:54 +0200)] 
Remove gradle usage from Javadoc gen script (#3386)

### Motivation

`javadoc-gen.sh` still uses Gradle for master branch.

### Changes

- Remove gradle and always use maven

Failure check: https://github.com/apache/bookkeeper/runs/7211295355?check_suite_focus=true

4 weeks agoMetrics: add journalCbQueueLatency (#3364)
LinChen [Thu, 7 Jul 2022 07:52:00 +0000 (15:52 +0800)] 
Metrics: add journalCbQueueLatency (#3364)

4 weeks agoFix dead link and add CI check (#3269)
Kirs [Wed, 6 Jul 2022 11:18:24 +0000 (19:18 +0800)] 
Fix dead link and add CI check (#3269)

* Fix dead link and add CI check

* fix dead link
exclude license checks for json files

* Run CI when only md files have been modified

* fix dead link

* fix dead link

* Ignore some link checks

* Ignore some link checks

* Ignore some link checks

* Ignore some link checks

* fix link checks

* ignore some links check

* ignore some links check

* ignore some links check

* fix dead link

* fix dead link

* fix dead link

* ignore some link

* fix deadlink

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* ignore some unknown link

* fix dead link

* fix dead link

* fix dead link

* fix dead link

* ignore some link

* ignore some link

* ignore some link

* fix dead link

* fix dead link

* rename job name

* fix config error

* ignore daxue link

4 weeks agostage 2.2: (#3380)
StevenLuMT [Wed, 6 Jul 2022 06:30:38 +0000 (14:30 +0800)] 
stage 2.2: (#3380)

if old bookeeper node want to upgrade this cookie
add a funtion in cookie_generate for update indexDirs

Co-authored-by: lushiji <lushiji@didiglobal.com>
4 weeks agoFix: NPE in RackawareEnsemblePlacementPolicyImpl logged by AutoRecovery (#3350)
Andrey Yegorov [Wed, 6 Jul 2022 00:57:52 +0000 (17:57 -0700)] 
Fix: NPE in RackawareEnsemblePlacementPolicyImpl logged by AutoRecovery (#3350)

### Motivation

AR keeps on logging NPE during rereplication:

```
[ReplicationWorker] WARN  org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl - Received exception while trying to get network location of bookie: ...
java.lang.NullPointerException: null
```

The NPE seems to be easily avoidable, no need to throw exception and log it just to ignore.

### Changes

Added check for bookie being present in the knownBookies map before accessing it.

4 weeks agoBP-47 (task7): DbLedgerStorage add direct entry logger support (#3366)
Hang Chen [Mon, 4 Jul 2022 15:34:34 +0000 (23:34 +0800)] 
BP-47 (task7): DbLedgerStorage add direct entry logger support (#3366)

5 weeks agoFix the Intellij key nouns. (#3381)
liuzhuang2017 [Sat, 2 Jul 2022 06:02:07 +0000 (14:02 +0800)] 
Fix the Intellij key nouns. (#3381)

5 weeks agoadd and check indexDirs in Cookie meta (#3372)
StevenLuMT [Fri, 1 Jul 2022 06:13:46 +0000 (14:13 +0800)] 
add and check indexDirs in Cookie meta (#3372)

Co-authored-by: lushiji <lushiji@didiglobal.com>
5 weeks agouse estimateMaxDirectMemory instead of maxDirectMemory (#3375)
Hang Chen [Thu, 30 Jun 2022 15:19:53 +0000 (23:19 +0800)] 
use estimateMaxDirectMemory instead of maxDirectMemory (#3375)

### Motivation
https://github.com/apache/bookkeeper/pull/2989

### Modification
Use estimateMaxDirectMemory instead of maxDirectMemory

5 weeks agoFix a typo in bk_server.conf (#3370)
ZhangJian He [Thu, 30 Jun 2022 06:02:23 +0000 (14:02 +0800)] 
Fix a typo in bk_server.conf (#3370)

5 weeks agoconvert testcase to junit 4 (#3371)
StevenLuMT [Tue, 28 Jun 2022 17:58:39 +0000 (01:58 +0800)] 
convert testcase to junit 4 (#3371)

* Basic function: directories to store index files

* add test case for indexDirs fix

* checkstyle fixed

* use junit 5 api to run testcase

* convert this testcase to junit 4

Co-authored-by: lushiji <lushiji@didiglobal.com>
5 weeks agoBasic function: directories to store index files (#3352)
StevenLuMT [Tue, 28 Jun 2022 11:23:06 +0000 (19:23 +0800)] 
Basic function: directories to store index files (#3352)

Co-authored-by: lushiji <lushiji@didiglobal.com>
5 weeks agoFix bookie CI test not run (#3367)
Hang Chen [Tue, 28 Jun 2022 01:02:19 +0000 (09:02 +0800)] 
Fix bookie CI test not run (#3367)

### Motivation
The `bookie-tests` CI runs the following tests.
```
- name: Run bookie tests
        run: mvn -B -nsu -pl bookkeeper-server test -Dtest="org.apache.bookkeeper.bookie.*Test" -DfailIfNoTests=false -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
```

For the `remaining-tests` CI runs the following tests.
```bash
- name: Run remaining tests
        run: mvn -B -nsu -am -pl bookkeeper-server clean install test -Dtest="!org.apache.bookkeeper.client.**,!org.apache.bookkeeper.bookie.**,!org.apache.bookkeeper.replication.**,!org.apache.bookkeeper.tls.**" -DfailIfNoTests=false -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
```

The problem is that `bookie-tests` only run tests with the name `org.apache.bookkeeper.bookie.*Test`, however, the `remaining-tests` exclude tests with name `org.apache.bookkeeper.bookie.**`. It leads to a lot of tests in `org.apache.bookkeeper.bookie.*` package doesn't run at all in CI.

### Modification
Turn on `org.apache.bookkeeper.bookie.**` tests in `bookie-tests` CI.

I have been testing on my laptop, there are many tests can't pass, I may need more time to make those failed tests to be passed.

The following tests need to fixed.
- [x] org.apache.bookkeeper.bookie.storage.ldb.ConversionTest.test
- [x] org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageTest.testLimboStateThrowsInLimboWhenNoEntry
- [x] org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageTest.testLimboStateSucceedsWhenFenced
- [x] org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageTest.testLimboStateSucceedsWhenInLimboButHasEntry
- [x] org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageTest.testLimboStateThrowsInLimboWhenNotFenced
- [x] org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorageTest.testStorageStateFlags

6 weeks agoBP-47 (task6): Direct I/O entrylogger support (#3263)
Hang Chen [Thu, 23 Jun 2022 02:19:15 +0000 (10:19 +0800)] 
BP-47 (task6): Direct I/O entrylogger support (#3263)

* Direct I/O entry log support

The implementation uses JNI to do direct I/O to files via posix
syscalls. Fallocate is used if running on linux, otherwise this is
skipped (at the cost of more filesystem operates during writing).

There are two calls to write, writeAt and writeDelimited. I expect
writeAt to be used for the entrylog headers, which entries will go
through writeDelimited. In both cases, the calls may return before the
syscalls occur. #flush() needs to be called to ensure things are
actually written.

The entry log format isn't much changed from what is used by the
existing entrylogger. The biggest difference is the padding. Direct
I/O must write in aligned blocked. The size of the alignment varies by
machine configuration, but 4K is a safe bet on most. As it is unlikely
that entry data will land exactly on the alignment boundary, we need
to add padding to writes. The existing entry logger has been changed
to take this padding into account. When read as a signed int/long/byte
the padding will aways parse to a negative value, which distinguishes
it from valid entry data (the entry size will always be positive) and
also from preallocated space (which is always 0).

Another difference in the format is that the header is now 4K rather
than 1K. Again, this is to allow aligned rights. No changes are
necessary to allow the existing entry logger to deal with the header
change, as we create a dummy entry in the extra header space that the
existing entry logger already knows to ignore.

To enable, set dbStorage_directIOEntryLogger=true in the configuration.

(cherry picked from commit 09a1c81b6894d4398c5a59e54cc18784550badb5)

* format code

* format code

* fix spotbugs check

* fix jnilib not found

* fix jnilib not found

* address comments

* fix so lib not found

* address comments

* format code

* add compat test

* add compat test

* address comments

* fix findbugs failed

* format code

Co-authored-by: Ivan Kelly <ikelly@splunk.com>
6 weeks agoupgrade groovy from 2.5.17 to 3.0.11 to fix CVE-2019-11358(7.5) (#3346)
Hang Chen [Wed, 22 Jun 2022 13:09:40 +0000 (21:09 +0800)] 
upgrade groovy from 2.5.17 to 3.0.11 to fix CVE-2019-11358(7.5) (#3346)

6 weeks agoAutorecovery to rereplicate empty ledgers (#3239)
Andrey Yegorov [Tue, 21 Jun 2022 09:44:38 +0000 (02:44 -0700)] 
Autorecovery to rereplicate empty ledgers (#3239)

6 weeks agoreduce unnecessary checkpoints (#3341)
LinChen [Mon, 20 Jun 2022 07:01:47 +0000 (15:01 +0800)] 
reduce unnecessary checkpoints (#3341)

6 weeks agofix: broken link of code guide in README (#3347)
Max Xu [Mon, 20 Jun 2022 06:43:21 +0000 (14:43 +0800)] 
fix: broken link of code guide in README (#3347)

6 weeks agoallow build on jdk17 (#3343)
ZhangJian He [Sun, 19 Jun 2022 18:33:02 +0000 (02:33 +0800)] 
allow build on jdk17 (#3343)

7 weeks agoRemove unused value assign in BookKeeperTestClient (#3331)
LeKe [Sun, 19 Jun 2022 14:54:05 +0000 (22:54 +0800)] 
Remove unused value assign in BookKeeperTestClient (#3331)

7 weeks agoRemove gradle build script (#3340)
ZhangJian He [Fri, 17 Jun 2022 00:39:28 +0000 (08:39 +0800)] 
Remove gradle build script (#3340)

### Motivation
Remove gradle build script
See
https://lists.apache.org/thread/k3pdlpbvkzt061338fkxtppq3z70nk03
https://lists.apache.org/thread/os28qm4t0kf74q2k3lzsmnnpr0lnm6zc
### Changes
- delete gradle build script
- add `netty-buffer` dependency on module `distributedlog-core-shaded-test` to fix broken master tests.

7 weeks agoRemove gradle from workflow (#3314)
ZhangJian He [Thu, 16 Jun 2022 08:11:58 +0000 (16:11 +0800)] 
Remove gradle from workflow (#3314)

* Remove gradle from workflow

* add dep

7 weeks agoApply the backpressure changes on the V2 requests (#3324)
Yong Zhang [Thu, 16 Jun 2022 00:17:19 +0000 (08:17 +0800)] 
Apply the backpressure changes on the V2 requests (#3324)

---

*Motivation*

If one bookie is slow (not down, just slow), the BK client
will the acks to the user that the entries are written after
the first 2 acks. In the meantime, it will keep waiting for
the 3rd bookie to respond. If the bookie responds within the
timeout, the entries can now be dropped from memory, otherwise
the write will timeout internally and it will get replayed
to a new bookie.

In the V3 request, we have [server-side backpressure](https://github.com/apache/bookkeeper/pull/1410)
to impact the client-side behaviors. We should apply the same
changes to the V2 request. That would help this [issue](https://github.com/apache/pulsar/issues/14861)
to be resolved.

*Modification*

- Apply the change https://github.com/apache/bookkeeper/pull/1410 to V2 protocol

Descriptions of the changes in this PR:

7 weeks ago Fix the V2 AddRequest object leak issue (#3323)
Yong Zhang [Mon, 13 Jun 2022 07:27:51 +0000 (15:27 +0800)] 
 Fix the V2 AddRequest object leak issue (#3323)

---

**Motivation**

If the request is a V2 add request, we retained the data's
reference when creating the AddRequest object. To avoid the
object leak, we need to release the reference if we met
any errors before sending it.

8 weeks agoAdd Pravega as a Bookkeeper use case. (#3328)
Raúl Gracia [Fri, 10 Jun 2022 09:30:49 +0000 (11:30 +0200)] 
Add Pravega as a Bookkeeper use case. (#3328)

8 weeks ago[Client] Deduplicate error log for SSLException (#3320)
Michael Marshall [Thu, 9 Jun 2022 05:08:07 +0000 (00:08 -0500)] 
[Client] Deduplicate error log for SSLException (#3320)

### Motivation

While testing #3310, I noticed that the `PerChannelBookieClient#exceptionCaught` logic contains redundant logs when the exception is an `SSLException`. This PR removes a redundant log from client.

Based on reading through the rest of the method, this should be a trivial change with no other side effects. My one question is how closing a channel and closing the context differ. Technically, returning early skips the `ctx.close()`, which could change the behavior.

### Changes

* Return early to prevent a redundant log

8 weeks agoFix the PendingAddOp is not recycled when LedgerHandler closed (#3321)
Yong Zhang [Wed, 8 Jun 2022 10:02:32 +0000 (18:02 +0800)] 
Fix the PendingAddOp is not recycled when LedgerHandler closed (#3321)

---

**Motivation**

When adding an entry into a bookie, the entry data lifecycle
is handled by the bookie client. The data buffer will be
released after receiving a corresponding response from the
bookie server. So the user doesn't care about the entry
buffer releasing.
But when the ledgerHandler is closed, the PendingAddOp is not
recycled which leads to the data buffer never being released.
We should release that after the callback executed.

8 weeks agoTry to use Java9 CRC32C when JNI based CRC is not available (#3309)
Matteo Merli [Wed, 8 Jun 2022 01:26:50 +0000 (18:26 -0700)] 
Try to use Java9 CRC32C when JNI based CRC is not available (#3309)

* Try to use Java9 CRC32C when JNI based CRC is not available

* Fixed shading test dependency

* Fixed dlog shading deps

2 months agoEnable maven batch mode to reduce log in mac and windows workflow (#3318)
TingTing Wang [Tue, 7 Jun 2022 12:51:53 +0000 (20:51 +0800)] 
Enable maven batch mode to reduce log in mac and windows workflow (#3318)

2 months agoFix maven javadoc generate (#3317)
ZhangJian He [Tue, 7 Jun 2022 10:54:02 +0000 (18:54 +0800)] 
Fix maven javadoc generate (#3317)

### Motivation
Currently, the maven javadoc generate is broken both on master and branch-4.14

### Changes
- fix the javadoc generate, the gradle script has already use `delombok`
- delete unneeded `package-info` file
- delete unused dependency `spotbugs-annotations`, it's not worth to open other pr(IMO)

2 months agoConsider consider Bookie ID when validating the Cookie. (#3308)
Raúl Gracia [Tue, 7 Jun 2022 10:53:54 +0000 (12:53 +0200)] 
Consider consider Bookie ID when validating the Cookie. (#3308)

Signed-off-by: Raúl Gracia <raul.gracia@emc.com>
2 months agoadd missing link to paxos algorithm (#3298)
Vladislav Khakin [Tue, 7 Jun 2022 03:14:34 +0000 (06:14 +0300)] 
add missing link to paxos algorithm (#3298)

Fixes #3293

2 months agoBump apache version from 19 to 26 (#3315)
ZhangJian He [Mon, 6 Jun 2022 07:07:51 +0000 (15:07 +0800)] 
Bump apache version from 19 to 26 (#3315)

2 months agoClose journal channel in testJunkEndedJournal (#3307)
ZhangJian He [Mon, 6 Jun 2022 05:41:47 +0000 (13:41 +0800)] 
Close journal channel in testJunkEndedJournal (#3307)

2 months agoBump grpc from 1.45.1 to 1.47.0, solve dependency check FP (#3305)
ZhangJian He [Sun, 5 Jun 2022 07:29:26 +0000 (15:29 +0800)] 
Bump grpc from 1.45.1 to 1.47.0, solve dependency check FP (#3305)

* Bump grpc from 1.45.1 to 1.47.0

* add dependency check suppression

2 months ago[Client] Switch to rely on Netty for Hostname Verification (#3310)
Michael Marshall [Sun, 5 Jun 2022 06:39:35 +0000 (01:39 -0500)] 
[Client] Switch to rely on Netty for Hostname Verification (#3310)

### Motivation

Currently, we initiate hostname verification for the Bookkeeper Client in the `PerChannelBookieClient` class. In order to simplify the code, I propose that we refactor the client so it relies on Netty, its SslHandler/SslEngine, and the JVM, to perform the hostname verification.

When HTTPS is configured as the endpoint verification algorithm, it uses [RFC 2818](https://datatracker.ietf.org/doc/html/rfc2818) to perform hostname verification. This is defined by the Java Security Standard Algorithm Names documentation for JDK versions 8, 11, and 17. Here are the official docs:

* https://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html
* https://docs.oracle.com/en/java/javase/11/docs/specs/security/standard-names.html
* https://docs.oracle.com/en/java/javase/17/docs/specs/security/standard-names.html

### Changes

* Rely on Netty and the SslEngine to perform hostname verification. With this change, CN matching is now deprecated, which brings the bookkeeper client in alignment with RFC 2818.
* Add new method to the `SecurityHandlerFactory` interface. It is named `newTLSHandler` and takes the `host` and `port` of the remote peer when creating a new SslEngine. To ensure backwards compatibility, the default implementation will call the original method. Note that the remote host and port are only needed when a client is using them for hostname verification.

2 months agoClean up javadoc mojos execution (#2906)
Fangbin Sun [Fri, 3 Jun 2022 05:23:06 +0000 (13:23 +0800)] 
Clean up javadoc mojos execution (#2906)

Descriptions of the changes in this PR:

### Motivation

Fixes #1514.

### Changes

1. Configuring maven-javadoc-plugin with `detectOfflineLinks=false` eliminates the Maven invocations (refer to: [https://issues.redhat.com/browse/ISPN-8629](https://issues.redhat.com/browse/ISPN-8629)).
2. Use option `notimestamp` to replace `additionalparam` since the latter is not allowed from version `3.x`.

2 months ago[build] Support windows build (#3303)
ZhangJian He [Thu, 2 Jun 2022 15:05:22 +0000 (23:05 +0800)] 
[build] Support windows build (#3303)

### Motivation
Support windows build to let more people easily contributed in this project. Test on my windows10 and windows11 PC.
I think that Windows now can be used for development, but not for production.
### Changes
- add some win32 compat code
- add windows build check
### owasp ci failure
both `google-http-client-gson` and `maven-settings` are not related to this PR

2 months ago[ci] [build] add macos build check (#3301)
ZhangJian He [Wed, 1 Jun 2022 11:09:14 +0000 (19:09 +0800)] 
[ci] [build] add macos build check (#3301)

2 months ago[build] [windows] fix broken master code compile error strerror_s (#3300)
ZhangJian He [Tue, 31 May 2022 02:06:21 +0000 (10:06 +0800)] 
[build] [windows] fix broken master code compile error strerror_s (#3300)

### Motivation
master compile is broken by #2549

### Changes

Add windows `#ifdef _WIN32` directive on `define`. Test on my m1 mac and windows 10.

2 months agoFix javadoc and build warnings (#3297)
Vladislav Khakin [Mon, 30 May 2022 23:47:45 +0000 (02:47 +0300)] 
Fix javadoc and build warnings (#3297)

Fix #472 #2155

2 months ago[build] support circe-checksum and cpu-affinity build on Windows (#2549)
magiccao [Mon, 30 May 2022 11:03:38 +0000 (19:03 +0800)] 
[build] support circe-checksum and cpu-affinity build on Windows (#2549)

Co-authored-by: dongcao <dongcao@trip.com>
2 months ago[build] misc bump several maven plugin version (#3294)
ZhangJian He [Mon, 30 May 2022 06:21:55 +0000 (14:21 +0800)] 
[build] misc bump several maven plugin version (#3294)

### Motivation
Keep plugin version update. I am trying to make bookkeeper can run on windows(more developers can join this project). I believe update some maven plugin versions can help a little.

### Changes
- bump jacoco maven plugin from 0.8.0 to 0.8.8
- bump maven-clean-plugin from 2.5 to 3.2.0
- bump maven-compiler version to 3.10.1
- bump maven-jar-plugin from 2.4 to 3.2.2
- bump nar-maven-plugin from 3.5.2 to 3.10.1
- delete a unnecessary dependency, it's already defined in parent pom

### ci failure
both `google-http-client-gson` and `maven-settings` are not related to this PR