Alexey Romanenko [Thu, 19 May 2022 08:32:09 +0000 (10:32 +0200)]
Merge pull request #17572: [BEAM-14428] I/O, community, and contribute pages improvements
Yichi Zhang [Wed, 18 May 2022 19:29:38 +0000 (12:29 -0700)]
Update release-guide.md
dependabot[bot] [Wed, 18 May 2022 17:19:00 +0000 (10:19 -0700)]
Bump github.com/lib/pq from 1.10.5 to 1.10.6 in /sdks (#17691)
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.5 to 1.10.6.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.5...v1.10.6)
---
updated-dependencies:
- dependency-name: github.com/lib/pq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Danny McCormick [Wed, 18 May 2022 17:18:29 +0000 (13:18 -0400)]
Fix a few small linting bugs (#17695)
johnjcasey [Wed, 18 May 2022 16:15:37 +0000 (12:15 -0400)]
[BEAM-10529] update KafkaIO Xlang integration test to publish and receive null keys (#17319)
* [BEAM-10529] update test to publish and receive null keys
* [BEAM-10529] add test with a populated key to kafka xlang_kafkaio_it_test.py
Yi Hu [Wed, 18 May 2022 15:23:21 +0000 (11:23 -0400)]
[BEAM-14474] Suppress 'Mean of empty slice' Runtime Warning in dataframe unit test (#17682)
* [BEAM-14474] Suppress 'Mean of empty slice' Runtime Warning in dataframe unit test
* use catch_warnings context manager
Brian Hulette [Tue, 17 May 2022 16:49:13 +0000 (09:49 -0700)]
[BEAM-14467] Fix bug where run_pytest.sh does not elevate errors raised in no_xdist tests (#17687)
tvalentyn [Tue, 17 May 2022 16:16:53 +0000 (18:16 +0200)]
[BEAM-12000] Update programming-guide.md (#17679)
Darkhan Nausharipov [Tue, 17 May 2022 14:41:32 +0000 (20:41 +0600)]
README update for the Docker Error 255 during Website launch on Apple Silicon (#17456)
Co-authored-by: Danny McCormick <dannymccormick@google.com>
Danny McCormick [Tue, 17 May 2022 14:16:31 +0000 (09:16 -0500)]
[BEAM-14441] Automatically assign issue labels based on responses to template (#17661)
Moritz Mack [Tue, 17 May 2022 13:58:05 +0000 (15:58 +0200)]
Merge pull request #17692 from aromanenko-dev/BEAM-14035-checkstyle-fix
[BEAM-14035] Fix checkstyle issue
Alexey Romanenko [Tue, 17 May 2022 12:52:36 +0000 (14:52 +0200)]
[BEAM-14035] Fix checkstyle issue
Alexey Romanenko [Tue, 17 May 2022 11:56:50 +0000 (13:56 +0200)]
Merge pull request #17662:[BEAM-14334] Remove remaining forkEvery 1 from all Spark tests and stop mixing unit tests with runner validations.
Vladislav Chunikhin [Tue, 17 May 2022 09:05:58 +0000 (12:05 +0300)]
Merge pull request #17678 from [BEAM-14460] [Playground] WIP. Fix error during getting the graph for java SDK.
Brian Hulette [Mon, 16 May 2022 22:55:35 +0000 (15:55 -0700)]
[BEAM-14411] Re-enable TypecodersTest, fix most issues (#17547)
* Re-enable TypecodersTest, fix most issues
* yapf
* lint
Yichi Zhang [Mon, 16 May 2022 22:54:57 +0000 (15:54 -0700)]
Update release-guide.md
Yichi Zhang [Mon, 16 May 2022 22:47:11 +0000 (15:47 -0700)]
Update release-guide.md
David Huntsperger [Mon, 16 May 2022 19:34:43 +0000 (12:34 -0700)]
Adding eslint and lint configuration to TypeScript SDK (#17676)
Added eslint and TypeScript linting config to the TypeScript SDK. You can run it with npm run lint, but it's not integrated with the build step, so won't fail the build -- just informational.
Minbo Bae [Mon, 16 May 2022 19:32:17 +0000 (12:32 -0700)]
Merge pull request #17520 from BEAM-12356 Close DatasetService leaked with getTable
bullet03 [Mon, 16 May 2022 19:01:24 +0000 (01:01 +0600)]
Add Akvelon to case-studies (#17611)
* add image and md file to case-studies
* change text solid for deep
Damon Douglas [Mon, 16 May 2022 18:02:39 +0000 (18:02 +0000)]
[BEAM-14035 ] Implement BigQuerySchema Read/Write TransformProvider (#17607)
Jack McCluskey [Mon, 16 May 2022 17:46:20 +0000 (13:46 -0400)]
[BEAM-14440] Add basic fuzz tests to the coders package (#17587)
* UTF8 fuzz test
* Ints fuzz tests
* Bytes fuzz tests
* Double fuzz test
* Move fuzz tests to standalone test file
Jack McCluskey [Mon, 16 May 2022 16:33:33 +0000 (12:33 -0400)]
[BEAM-14473] Throw error if using globally windowed, unbounded side input (#17681)
Ryan Thompson [Mon, 16 May 2022 14:30:14 +0000 (10:30 -0400)]
[BEAM-13982] Added output of logging for python E2E pytests (#17637)
Lukasz Cwik [Mon, 16 May 2022 14:08:44 +0000 (07:08 -0700)]
[BEAM-13015] Update the SDK harness grouping table to be memory bounded based upon the amount of assigned cache memory and to use an LRU eviction policy. (#17327)
* [BEAM-13015] Update the grouping table to be memory bounded based upon amount of assigned cache memory and also use an LRU policy for evicting entries from the table.
* fixup! checkstyle
* fixup! Address PR comments.
Robert Burke [Sat, 14 May 2022 22:49:04 +0000 (15:49 -0700)]
[BEAM-14470] Use Generic Registrations in loadtests. (#17673)
Danny McCormick [Fri, 13 May 2022 23:04:58 +0000 (19:04 -0400)]
[BEAM-14371] (and BEAM-14372) - enable a couple staticchecks (#17670)
Robert Bradshaw [Fri, 13 May 2022 22:00:42 +0000 (15:00 -0700)]
Add some auto-starting runners to the typescript SDK. (#17580)
Adds out-of-the-box support for FlinkRunner, DataflowRunner, and the Python Universal Local runner. Also adds a DefaultRunner which chooses between the DirectRunner and the ULR depending on the properties of the pipeline.
Jack McCluskey [Fri, 13 May 2022 21:20:49 +0000 (17:20 -0400)]
[BEAM-14469] Allow nil primary returns from TrySplit in a single-window context (#17667)
Ryan Thompson [Fri, 13 May 2022 19:28:22 +0000 (15:28 -0400)]
[BEAM-14014] Support impersonation credentials in dataflow runner (#17244)
Heejong Lee [Fri, 13 May 2022 18:37:31 +0000 (11:37 -0700)]
Merge pull request #17605 from ihji/BEAM-14455
[BEAM-14455] Add UUID to sub-schemas for PythonExternalTransform
Marco Robles [Fri, 13 May 2022 17:50:53 +0000 (12:50 -0500)]
Merge pull request #17365 from [BEAM-12482] Update Schema Destination during Bigquery load job when using temporary tables using zeroloadjob
* Added a zero row job to bigquery writeTable
* added unit test
* add sdf for update schema destination
* add loadjob with inputstreamcontent parameter
* add updateschemadest with zeroloadjob before copying to dest
* add test for write temp tables
* add nullness supress warning
* remove unnecesary variable and use global variable for updateschemadestination-dofn
Co-authored-by: Miguel Anzo <miguel.anzo@wizeline.com>
Heejong Lee [Fri, 13 May 2022 17:29:55 +0000 (10:29 -0700)]
Merge pull request #17608 from ihji/BEAM-14430
[BEAM-14430] Adding a logical type support for Python callables to Row schema
Robert Bradshaw [Fri, 13 May 2022 16:23:25 +0000 (09:23 -0700)]
Merge pull request #17653 Revert "Better test assertion."
Jack McCluskey [Fri, 13 May 2022 16:16:31 +0000 (12:16 -0400)]
[BEAM-14465] Reduce DefaultS3ClientBuilderFactory logging to debug level (#17645)
dependabot[bot] [Fri, 13 May 2022 16:14:26 +0000 (09:14 -0700)]
Bump github.com/spf13/cobra from 1.3.0 to 1.4.0 in /sdks (#17647)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Changelog](https://github.com/spf13/cobra/blob/v1.4.0/CHANGELOG.md)
- [Commits](https://github.com/spf13/cobra/compare/v1.3.0...v1.4.0)
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Moritz Mack [Fri, 13 May 2022 14:16:56 +0000 (16:16 +0200)]
[BEAM-14334] Remove remaining forkEvery 1 from all Spark tests and stop mixing unit tests with runner validations.
Brian Hulette [Fri, 13 May 2022 06:56:21 +0000 (23:56 -0700)]
Drop dataclasses requirement, we only support python 3.7+ (#17640)
tvalentyn [Fri, 13 May 2022 06:54:11 +0000 (08:54 +0200)]
Revert "Better test assertion. (#17551)"
This reverts commit
dc20185f6af2e862bd62f9df7b835593a466c5f1.
Danny McCormick [Fri, 13 May 2022 02:12:53 +0000 (22:12 -0400)]
[BEAM-14347] Add function for simple function registration (#17650)
Heejong Lee [Fri, 13 May 2022 01:02:09 +0000 (18:02 -0700)]
add comment
Ahmet Altay [Thu, 12 May 2022 23:34:49 +0000 (16:34 -0700)]
Merge pull request #17631 from chamikaramj/update_release_md
Heejong Lee [Thu, 12 May 2022 23:12:46 +0000 (16:12 -0700)]
put a default type hint for PythonCallableSource
dependabot[bot] [Thu, 12 May 2022 22:25:59 +0000 (15:25 -0700)]
Bump cloud.google.com/go/bigquery from 1.28.0 to 1.32.0 in /sdks (#17625)
Bumps [cloud.google.com/go/bigquery](https://github.com/googleapis/google-cloud-go) from 1.28.0 to 1.32.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.28.0...spanner/v1.32.0)
---
updated-dependencies:
- dependency-name: cloud.google.com/go/bigquery
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Heejong Lee [Thu, 12 May 2022 22:10:50 +0000 (15:10 -0700)]
add micros_instant urn
Heejong Lee [Thu, 12 May 2022 21:46:24 +0000 (14:46 -0700)]
move logical types def
bullet03 [Thu, 12 May 2022 20:48:41 +0000 (02:48 +0600)]
Merge pull request #17408 from [BEAM-14312] [Website] change section order, move socials to footer
* [BEAM-14312] [Website] change section order, move socials to footer
* [BEAM-14312] [Website] change margin
* [BEAM-14312] [Website] change link
* [BEAM-14312] [Website] delete section, change title and links of the pillars
* [BEAM-14312] [Website] add pipelines section
* [BEAM-14312] [Website] change image in graphic section
* [BEAM-14312] [Website] delete width of 95%, change flink link
* [BEAM-14312] [Website] move css style, delete unused html, optimized logos size
* [BEAM-14312] [Website] substitute images, fix css styles for pipelines and logos sections
* [BEAM-14312] [Website] add kinesis runner
* [BEAM-14312] [Website] add kinesis runner, add margin-right to pipelines-logos
* [BEAM-14312] [Website] change amazon kinesis logo
* [BEAM-14312] change structure of logos in the footer
* [BEAM-14312] add license header to linkedin.svg
* [BEAM-14312] change margin in quote section
dependabot[bot] [Thu, 12 May 2022 20:44:23 +0000 (13:44 -0700)]
Bump cloud.google.com/go/pubsub from 1.18.0 to 1.21.1 in /sdks (#17646)
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.18.0 to 1.21.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.18.0...pubsub/v1.21.1)
---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Pablo [Thu, 12 May 2022 20:38:55 +0000 (16:38 -0400)]
Merge pull request #17584 from [BEAM-14415] Exception handling tests and logging for partial failure BQIO
* [BEAM-14415] Exception handling tests and logging for partial failures in BQ IO
* fix DLQ integration test
* fix lint
* fix postcommit
* fix formatter
* Fixing tests and adding test info
* fix skipping tests
dependabot[bot] [Thu, 12 May 2022 20:06:09 +0000 (13:06 -0700)]
Bump github.com/lib/pq from 1.10.4 to 1.10.5 in /sdks (#17626)
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.4 to 1.10.5.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.4...v1.10.5)
---
updated-dependencies:
- dependency-name: github.com/lib/pq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dependabot[bot] [Thu, 12 May 2022 20:05:11 +0000 (13:05 -0700)]
Bump github.com/testcontainers/testcontainers-go in /sdks (#17627)
Bumps [github.com/testcontainers/testcontainers-go](https://github.com/testcontainers/testcontainers-go) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/testcontainers/testcontainers-go/releases)
- [Commits](https://github.com/testcontainers/testcontainers-go/compare/v0.12.0...v0.13.0)
---
updated-dependencies:
- dependency-name: github.com/testcontainers/testcontainers-go
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dependabot[bot] [Thu, 12 May 2022 20:04:42 +0000 (13:04 -0700)]
Bump github.com/google/go-cmp from 0.5.7 to 0.5.8 in /sdks (#17628)
Bumps [github.com/google/go-cmp](https://github.com/google/go-cmp) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/google/go-cmp/releases)
- [Commits](https://github.com/google/go-cmp/compare/v0.5.7...v0.5.8)
---
updated-dependencies:
- dependency-name: github.com/google/go-cmp
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Robert Bradshaw [Thu, 12 May 2022 20:02:14 +0000 (13:02 -0700)]
Better test assertion. (#17551)
Danny McCormick [Thu, 12 May 2022 20:00:55 +0000 (16:00 -0400)]
[BEAM-14347] Add generic registration feature to CHANGES (#17643)
Aizhamal Nurmamat kyzy [Thu, 12 May 2022 17:13:12 +0000 (10:13 -0700)]
Merge pull request #17588 from damccorm/users/damccorm/issueTemplates
[BEAM-14442] Add GitHub issue templates
Yichi Zhang [Thu, 12 May 2022 17:06:16 +0000 (10:06 -0700)]
Revert "[BEAM-14429] Force java load test on dataflow runner v2 forceNumInitialBundles to 1 (#17576)" (#17609)
This reverts commit
178441dcf0a1e03611c46bdd12ab27907289e310.
Danny McCormick [Thu, 12 May 2022 14:37:27 +0000 (10:37 -0400)]
Trigger go precommits on go mod/sum changes (#17636)
Danny McCormick [Thu, 12 May 2022 13:30:51 +0000 (09:30 -0400)]
Pare down to fewer templates
Robert Burke [Thu, 12 May 2022 13:17:35 +0000 (06:17 -0700)]
Typo & link update (#17633)
Moritz Mack [Thu, 12 May 2022 13:08:19 +0000 (15:08 +0200)]
[BEAM-14334] Fix leakage of SparkContext in Spark runner tests to remove forkEvery 1 (#17406)
* [BEAM-14334] Fix leakage of SparkContext in Spark runner tests to remove forkEvery 1 and set provided SparkContext via SparkContextFactory to avoid losing it during a serde roundtrip in TestPipenline.
Heejong Lee [Thu, 12 May 2022 05:29:12 +0000 (22:29 -0700)]
fix lint errors
Heejong Lee [Thu, 12 May 2022 03:34:20 +0000 (20:34 -0700)]
add urn, type inference for PythonCallableSource
Kiley Sok [Wed, 11 May 2022 21:48:57 +0000 (14:48 -0700)]
[BEAM-13695] Add jamm jvm options to Java 11 (#17178)
* Add jamm jvm options to Java 11
* fix test comments
Danny McCormick [Wed, 11 May 2022 21:16:50 +0000 (17:16 -0400)]
Correctly route go dependency changes to go label (#17632)
Chamikara Jayalath [Wed, 11 May 2022 20:33:08 +0000 (13:33 -0700)]
Updates CHANGES.md to include some recently discovered known issues
Danny McCormick [Wed, 11 May 2022 19:59:21 +0000 (15:59 -0400)]
[BEAM-14347] Add some benchmarks for generic registration (#17613)
Danny McCormick [Wed, 11 May 2022 19:57:56 +0000 (15:57 -0400)]
Cut p4
Danny McCormick [Wed, 11 May 2022 19:48:56 +0000 (15:48 -0400)]
We don't need outage
Danny McCormick [Wed, 11 May 2022 19:45:03 +0000 (15:45 -0400)]
Ask for beam version + other dependencies
Yichi Zhang [Wed, 11 May 2022 19:08:45 +0000 (12:08 -0700)]
Remove python 3.6 postcommit from mass_comment.py (#17630)
Kenn Knowles [Wed, 11 May 2022 18:46:04 +0000 (11:46 -0700)]
Merge pull request #17519: [BEAM-14096] bump junit-quickcheck to 1.0
Danny McCormick [Wed, 11 May 2022 18:09:39 +0000 (14:09 -0400)]
[BEAM-12526] Add Dependabot (#17563)
* Create dependabot.yml
* Apache header
* See if dependabot will parse now
* Explanatory comment
* Reduce duplication
* Bug + style from feedback
* Change format to make dependabot happy
Igor Krasavin [Wed, 11 May 2022 16:23:23 +0000 (19:23 +0300)]
[BEAM-14081] [CdapIO] Add context classes for CDAP plugins (#17104)
* [BEAM-14048] Add ConfigWrapper for building CDAP PluginConfigs
* [BEAM-14048] Fix checkstyle
* [BEAM-14048] Fix warnings
* [BEAM-14048] Fix warnings
* [BEAM-14048] Fix warning
* [BEAM-14048] Fix warning
* [BEAM-14048] Remove unused dependencies
* [BEAM-14048] Add needed dependencies
* [BEAM-14048] Fix spotless
* [BEAM-14048] Fix typo
* [BEAM-14048] Use fori instead of stream
* [BEAM-14048] Suppress warning
* [BEAM-14048] Add used undeclared artifacts
* [BEAM-14048] Change dependencies to test
* Add context.
* Fix dependencies issue
* Add null annotation
* [BEAM-14048] Refactoring
* Add SuppressWarning.
* Fix style.
* Determine dependencies.
* [BEAM-14048] Use CDAP InstantiatorFactory for creating config objects
* [BEAM-14048] Suppress warning
* [BEAM-14081] Refactoring
* Update maven repo
* Update build.gradle
* [BEAM-14081] Refactoring
* [BEAM-14048] Use ServiceNow CDAP dependency from Maven central
* [BEAM-14048] Set macroFields
* [BEAM-14081] Fix javadoc
* [BEAM-14081] Make BatchContextImpl class abstract
Co-authored-by: vitaly.terentyev <vitaly.terentyev@akvelon.com>
Co-authored-by: Alex Kosolapov <alex.kosolapov@gmail.com>
Co-authored-by: Elizaveta Lomteva <elizaveta.lomteva@akvelon.com>
Co-authored-by: Elizaveta Lomteva <57974525+Lizzfox@users.noreply.github.com>
Jack McCluskey [Wed, 11 May 2022 15:58:53 +0000 (11:58 -0400)]
[BEAM-11104] Add self-checkpointing to CHANGES.md (#17612)
tvalentyn [Wed, 11 May 2022 13:58:59 +0000 (15:58 +0200)]
[BEAM-14396] Bump httplib2 upper bound. (#17602)
andoni-guzman [Wed, 11 May 2022 13:29:32 +0000 (08:29 -0500)]
[BEAM-5492] Python Dataflow integration tests should export the pipeline console output to Jenkins Test Result section (#17530)
Jack McCluskey [Wed, 11 May 2022 01:34:07 +0000 (21:34 -0400)]
[BEAM-11104] Add self-checkpointing integration test (#17590)
Danny McCormick [Tue, 10 May 2022 23:44:29 +0000 (19:44 -0400)]
[BEAM-14347] Rename registration package to register (#17603)
Heejong Lee [Fri, 6 May 2022 03:26:31 +0000 (20:26 -0700)]
[BEAM-14430] Adding a logical type support for Python callables to Row schema
Pablo [Tue, 10 May 2022 21:14:19 +0000 (17:14 -0400)]
Merge pull request #17601 from [BEAM-14447] Revert "Merge pull request #17517 from [BEAM-14383] Improve "FailedRo…
[BEAM-14447] Revert "Merge pull request #17517 from [BEAM-14383] Improve "FailedRo…
Heejong Lee [Tue, 10 May 2022 21:05:50 +0000 (14:05 -0700)]
[BEAM-14455] Add UUID to sub-schemas for PythonExternalTransform
Yichi Zhang [Tue, 10 May 2022 20:39:39 +0000 (13:39 -0700)]
[BEAM-14229] Fix SyntheticUnboundedSource duplication from checkpoint issue (#17600)
Pablo [Tue, 10 May 2022 17:28:39 +0000 (13:28 -0400)]
Merge pull request #17593 from [BEAM-14423] Add test cases for BigtableIO.BigtableWriterFn fails due to writeRecord
[BEAM-14423] Add test cases for BigtableIO.BigtableWriterFn fails due to writeRecord
Pablo [Tue, 10 May 2022 17:24:04 +0000 (13:24 -0400)]
Merge pull request #17592 from [BEAM-14448] add datastore exception handling test
[BEAM-14448] add datastore test
Pablo E [Tue, 10 May 2022 16:49:28 +0000 (09:49 -0700)]
Revert "Merge pull request #17517 from [BEAM-14383] Improve "FailedRows" errors returned by beam.io.WriteToBigQuery"
This reverts commit
358782006e1db86437b3bf61f910db12d654b1e0.
Etienne Chauchot [Tue, 10 May 2022 13:16:55 +0000 (15:16 +0200)]
Merge pull request #15679 from aromanenko-dev/BEAM-12918-tpcds-jenkins
Merge pull request #15679: [BEAM-12918] TPC-DS: add Jenkins jobs
Danny McCormick [Tue, 10 May 2022 00:05:24 +0000 (20:05 -0400)]
[BEAM-14347] Add generic registration for accumulators (#17579)
Elias Segundo Antonio [Mon, 9 May 2022 19:39:03 +0000 (14:39 -0500)]
[BEAM-14169] Add Credentials rotation cron job for clusters (#17383)
Add cron groovy job for credentials rotation. As Jenkins already has the `-xe` flag set while executing DSL jobs, if a command returns an error then the job will be stopped and marked as fail. Addded email notifications in case of failure and jobs were separated.
Co-authored-by: Elias Segundo <elias.segundo@luisrazo.local>
Yi Hu [Mon, 9 May 2022 17:11:32 +0000 (13:11 -0400)]
[BEAM-14423] Add test cases for BigtableIO.BigtableWriterFn fails due to writeRecord
Danny McCormick [Mon, 9 May 2022 18:49:02 +0000 (14:49 -0400)]
[BEAM-14347] Add generic registration functions for iters and emitters (#17574)
johnjcasey [Mon, 9 May 2022 18:33:28 +0000 (14:33 -0400)]
[BEAM-14448] add datastore test
Pablo [Mon, 9 May 2022 18:24:48 +0000 (14:24 -0400)]
Merge pull request #17577 from [BEAM-14435] Adding exception handling tests for SpannerIO write transform
* Adding exception handling tests for SpannerIO write transform
* fixup
Yichi Zhang [Mon, 9 May 2022 17:38:06 +0000 (10:38 -0700)]
[BEAM-14429] Force java load test on dataflow runner v2 forceNumInitialBundles to 1 (#17576)
Robert Burke [Mon, 9 May 2022 17:23:18 +0000 (10:23 -0700)]
[BEAM-14433] Improve Go split error message. (#17575)
Pablo [Mon, 9 May 2022 17:13:00 +0000 (13:13 -0400)]
Merge pull request #17555 from [BEAM-14417] Adding exception handling tests for JdbcIO.Write
* Adding exception handling tests for JdbcIO.Write
* Improving coordination in backoff test
johnjcasey [Mon, 9 May 2022 17:00:24 +0000 (13:00 -0400)]
Merge pull request #17565 from [BEAM-14413] add Kafka exception test cases
* [BEAM-10529] update test to publish and receive null keys
* BEAM-14413 start work on test cases
* [BEAM-14413] revert py change
* [BEAM-14413] add remaining kafka test cases
* [BEAM-14413] run spotless
* [BEAM-14413] fix race condition in tests
* [BEAM-14413] Rerun spotless
Pablo [Mon, 9 May 2022 16:52:15 +0000 (12:52 -0400)]
Merge pull request #17544 from [BEAM-14415] Exception handling tests for BQIO streaming inserts in Python
* Exception handling tests for BQIO streaming inserts in Python
* Adding non-retriable tests
* Added streaming retries
* Added streaming retries
* fix lint
* fix formatter
Jack McCluskey [Mon, 9 May 2022 16:26:13 +0000 (12:26 -0400)]
[BEAM-11104] Allow self-checkpointing SDFs to return without finishing their restriction (#17558)
Yi Hu [Mon, 9 May 2022 16:09:14 +0000 (12:09 -0400)]
Merge pull request #17559 from [BEAM-14423] Add exception injection tests for BigtableIO read in BigtableIOTest
* [BEAM-14423] Add exception injection tests for BigtableIO read in BigtableIOTest
* Test reading from bigtable when connection start fails
* Test reading from bigtable when split() fails
* Test reading from bigtable when advance() fails
* Also cleaned up deprecated Builder calls
* halfway commit
* Apply good practice for autovalue class
tvalentyn [Mon, 9 May 2022 15:48:55 +0000 (17:48 +0200)]
[BEAM-14439] [BEAM-12673] Add extra details to PubSub matcher errors #17586