Always reopen stream in FileUtils.copyLarge, RetryingInputStream. (#12307)
authorGian Merlino <gian@imply.io>
Sat, 5 Mar 2022 22:39:14 +0000 (14:39 -0800)
committerGitHub <noreply@github.com>
Sat, 5 Mar 2022 22:39:14 +0000 (14:39 -0800)
commit28f8bcce9b304fd123126073619ffd5f60f59f20
treefe988a1afc155770ab9abb9a835e509d7f6983c1
parent903174de2045449e12f94d00bdca09ecaf780e64
Always reopen stream in FileUtils.copyLarge, RetryingInputStream. (#12307)

* Always reopen stream in FileUtils.copyLarge, RetryingInputStream.

When an InputStream throws an exception from one of its read methods,
we should assume it's bad and reopen it.

The main changes here are:

- In FileUtils.copyLarge, replace InputStream with InputStreamSupplier.
- In RetryingInputStream, collapse retryCondition and resetCondition
  into a single condition. Also, make it required, since every usage
  is passing in a specific condition anyway.

* Test fixes.

* Fix read impl.
14 files changed:
core/src/main/java/org/apache/druid/data/input/InputEntity.java
core/src/main/java/org/apache/druid/data/input/RetryingInputEntity.java
core/src/main/java/org/apache/druid/data/input/impl/RetryingInputStream.java
core/src/main/java/org/apache/druid/data/input/impl/prefetch/FileFetcher.java
core/src/main/java/org/apache/druid/java/util/common/FileUtils.java
core/src/test/java/org/apache/druid/data/input/impl/RetryingInputStreamTest.java [new file with mode: 0644]
core/src/test/java/org/apache/druid/data/input/impl/prefetch/RetryingInputStreamTest.java [deleted file]
core/src/test/java/org/apache/druid/java/util/common/FileUtilsTest.java
extensions-core/parquet-extensions/src/test/java/org/apache/druid/data/input/parquet/ParquetReaderResourceLeakTest.java
extensions-core/s3-extensions/src/main/java/org/apache/druid/data/input/s3/S3Entity.java
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
extensions-core/s3-extensions/src/test/java/org/apache/druid/data/input/s3/S3InputSourceTest.java
indexing-service/src/main/java/org/apache/druid/indexing/input/DruidSegmentInputEntity.java
server/src/test/java/org/apache/druid/guice/QueryableModuleTest.java