Fil Karnicki [Thu, 10 Mar 2022 07:45:03 +0000 (08:45 +0100)]
[FLINK-26537][statefun] Allow disabling StatefulFunctionsConfigValidator validation for classloader.parent-first-patterns.additional
Fil Karnicki [Wed, 2 Mar 2022 21:01:08 +0000 (22:01 +0100)]
[FLINK-25866][statefun] Support additional TLS configuration
igalshilman [Fri, 25 Feb 2022 21:10:17 +0000 (22:10 +0100)]
[FLINK-26378][sdk][py] Set caller to None for ingress messages.
This commit makes sure that the caller's address would be None for
messages that are coming from an ingress.
Galen Warren [Fri, 25 Feb 2022 16:03:17 +0000 (11:03 -0500)]
[FLINK-26375][statefun-golang-sdk] Fix Statefun Golang SDK to return nil from Context.Caller when there is no caller
Change handler.go to only populate statefunContext.caller when a caller is present in the invocation
Add unit tests
Galen Warren [Tue, 22 Feb 2022 22:30:26 +0000 (17:30 -0500)]
[FLINK-26340][statefun-golang-sdk] Add ability in Golang SDK to create new statefun.Context from existing one, but with a new underlying context.Context
Change sync.Mutex in statefunContext to be *sync.Mutex, to make it copyable, and update construction of statefunContext accordingly
Add WithContext to statefun.Context interface and implement on statefunContext
Add unit test
This closes #303
Till Rohrmann [Tue, 15 Feb 2022 10:29:22 +0000 (11:29 +0100)]
[FLINK-26153] Add update_playground_links.sh to update playground links
This commit adds a new update_playground_links.sh script that updates links to
Statefun's playground in various places.
This closes #302.
Till Rohrmann [Tue, 15 Feb 2022 10:29:06 +0000 (11:29 +0100)]
[hotfix] Fix typo in update_branch_version.sh
Till Rohrmann [Tue, 15 Feb 2022 08:53:51 +0000 (09:53 +0100)]
[hotfix] Update README.md links to point to release-3.2 playground version
Till Rohrmann [Thu, 10 Feb 2022 17:13:23 +0000 (18:13 +0100)]
[hotfix] Let Python README.md point to Statefun playground
Till Rohrmann [Thu, 10 Feb 2022 17:10:07 +0000 (18:10 +0100)]
[FLINK-25898] Add README.md to statefun-sdk-js
This closes #301.
Galen Warren [Wed, 9 Feb 2022 17:24:38 +0000 (12:24 -0500)]
[FLINK-25933][statefun] Add support for the async HTTP transport in request-reply function builder
Create StatefulFunctionBuilder base class for all function builders
Create AsyncRequestReplyFunctionBuilder to build async functions
Refactor existing RequestReplyFunctionBuilder to extend StatefulFunctionBuilder
Create new factory methods on StatefulFunctionBuilder and deprecate old factory method
Modify SerializableHttpFunctionProvider to properly choose client factory based on HttpFunctionEndpointSpec
Update docs
Siddique Ahmad [Mon, 7 Feb 2022 11:02:51 +0000 (16:02 +0500)]
[hotfix][doc] Typo in link.
link was giving 404
igalshilman [Thu, 3 Feb 2022 08:47:44 +0000 (09:47 +0100)]
[FLINK-25936][java-sdk] Adjust the ConcurrentAddressScopedStorageTest to reflect the request-reply protocol
According to the request reply protocol, in case of a missing value: a registered state
but without an assosciated value (deleted, expired, or simply never set) the invocation
will contain an associated TypedValue with the field has_value = false, and might or might not
provide the type_name field.
This commit refelects that in the test.
igalshilman [Fri, 28 Jan 2022 20:16:49 +0000 (21:16 +0100)]
[FLINK-25936][java-sdk] Set value typename correctly in MutableTypeCell.
Before that commit, MutableTypeCell was assuming incorrectly
that the backend will send a TypedValue with a type_name field set even if the
value is missing (TypedValue.has_value = false). This is not the case, and hence
The type_name needs to be set explicitly.
martijnvisser [Tue, 1 Feb 2022 18:30:05 +0000 (19:30 +0100)]
[FLINK-25915][docs] Add Matomo tracking code to base layout
martijnvisser [Tue, 1 Feb 2022 18:29:06 +0000 (19:29 +0100)]
[FLINK-25915][docs] Remove Google Analytics implementation
Mingmin Xu [Thu, 27 Jan 2022 08:47:41 +0000 (00:47 -0800)]
[hotfix][doc] Remove invalid link in README
This closes #289.
Till Rohrmann [Wed, 26 Jan 2022 14:12:39 +0000 (15:12 +0100)]
[FLINK-25829] Skip deployment of statefun-sdk-js
This closes #286.
Till Rohrmann [Wed, 26 Jan 2022 14:01:26 +0000 (15:01 +0100)]
[FLINK-25828] Let build-stateful-functions.sh ignore javadoc and sources jar
This commit lets the build-stateful-functions.sh script ignore javadoc and sources jars that
can make the following copy commands to fail.
This closes #285.
Till Rohrmann [Tue, 25 Jan 2022 15:17:11 +0000 (16:17 +0100)]
[release] Add 3.2 release docs to docs/config.toml
Till Rohrmann [Tue, 25 Jan 2022 15:02:40 +0000 (16:02 +0100)]
[release] Update version to 3.3-SNAPSHOT
igalshilman [Mon, 24 Jan 2022 19:46:47 +0000 (20:46 +0100)]
[FLINK-25775][doc] Add Javascript SDK documentation for NodeJS
Till Rohrmann [Wed, 19 Jan 2022 13:46:48 +0000 (14:46 +0100)]
[FLINK-25708] Bump Flink dependency to 1.14.3
This closes #283.
Till Rohrmann [Mon, 27 Dec 2021 14:36:52 +0000 (15:36 +0100)]
[hotfix] Move junit and hamcrest-all test dependency to dependencyManagement
This commit sets the junit and hamcrest-all test dependency version and scope in the
dependencyManagement section of the root pom to unify all dependencies. This has the
purpose to avoid the usage of different testing libraries.
Galen Warren [Sun, 26 Dec 2021 16:24:23 +0000 (11:24 -0500)]
[FLINK-25197] Fix serialization issue in RequestReplyFunctionBuilder
This closes #282.
sjwiesman [Wed, 15 Dec 2021 19:34:10 +0000 (13:34 -0600)]
[hotfix] Add k8s HA to helm chart
This closes #279
sjwiesman [Wed, 15 Dec 2021 16:58:29 +0000 (10:58 -0600)]
[FLINK-25333] Disable auto uid generation and enable restoring from a savepoint with missing operators
sjwiesman [Wed, 15 Dec 2021 16:46:02 +0000 (10:46 -0600)]
[FLINK-25333] Manually set UIDs on router operator
igalshilman [Thu, 9 Dec 2021 14:02:43 +0000 (15:02 +0100)]
[FLINK-25308] Provide a custom remote module name
This commit adds a new flink-conf configuration key
that allows overriding the default remote module file name (module.yaml)
To provide a different name, for example prod.yaml that is located at /flink/usrlib/prod.yaml
One can add the following to their flink-conf.yaml:
statefun.remote.module-name: /flink/usrlib/prod.yaml
Till Rohrmann [Sat, 27 Nov 2021 18:28:12 +0000 (19:28 +0100)]
[hotfix][docs] Replace and with an in embedded.md
Till Rohrmann [Sat, 27 Nov 2021 16:46:51 +0000 (17:46 +0100)]
[hotfix][docs] Add missing be in logical.md documentation
igalshilman [Wed, 13 Oct 2021 19:00:11 +0000 (21:00 +0200)]
[FLINK-24534] Use the async transport if transport is not specified.
This commit sets the Netty based, asynchronous transport as the default transport
for endpoint definitions that did not specify a particular transport.
This closes #276
neoXfire [Tue, 12 Oct 2021 17:22:02 +0000 (19:22 +0200)]
[hotfix] Use JDK 8 Optional instead of shaded Optional class from Guava.
This closes #275
Igal Shilman [Mon, 27 Sep 2021 14:18:54 +0000 (16:18 +0200)]
[FLINK-22533] Add user defined metrics
This commits adds the ability to define custom metrics in the embedded SDK.
sjwiesman [Wed, 6 Oct 2021 23:31:28 +0000 (18:31 -0500)]
[FLINK-24464][core] Metrics sometimes report negative backlog
This closes #273
Igal Shilman [Thu, 23 Sep 2021 12:49:09 +0000 (14:49 +0200)]
[FLINK-24364][e2e] Add message cancellation embedded e2e case
This commit adds a message cancellation test case for the smoke e2e test.
The generation of message cancellation commands is disabled by default, and
only enabled for the embedded sdk e2e tests.
Igal Shilman [Wed, 22 Sep 2021 12:25:21 +0000 (14:25 +0200)]
[FLINK-24356][remote] Attach a cancellation token to a delayed message
This commit fixes a bug with the RequestReply function that previously ignored
the cancellation token sent from a remote SDK.
Ingo Bürk [Fri, 17 Sep 2021 08:42:37 +0000 (10:42 +0200)]
[FLINK-24322][js] Various improvements to the consistency and style
* Improve imports/exports and minor TS improvements
* Use Protobuf types where possible
* Seperate public api to a separate file
* Add jest explicitly
Igal Shilman [Fri, 10 Sep 2021 14:35:36 +0000 (16:35 +0200)]
[FLINK-24256][sdk] Add a JavaScript SDK
Evans Ye [Thu, 2 Sep 2021 19:22:56 +0000 (03:22 +0800)]
[FLINK-23985] Add document for adding new language SDK Smoke E2E
sjwiesman [Mon, 13 Sep 2021 22:35:27 +0000 (17:35 -0500)]
[hotfix][docs] fix documentation build
sjwiesman [Mon, 13 Sep 2021 20:05:27 +0000 (15:05 -0500)]
[hotfix][docs] drop unused io overview
sjwiesman [Wed, 1 Sep 2021 14:08:56 +0000 (09:08 -0500)]
[FLINK-24109] Correct egress string serialization
This closes #265
sjwiesman [Wed, 1 Sep 2021 14:21:16 +0000 (09:21 -0500)]
[FLINK-24110] Filter null ComponentJsonObjects
This closes #266
Igal Shilman [Thu, 2 Sep 2021 15:06:45 +0000 (17:06 +0200)]
[FLINK-24132] Remove directory-maven-plugin
The only usage of that plugin is at statefun-e2e-tests/pom.xml
and to only invoke two shell script. We can simply use a relative path
Since the shell scripts resolve their basedir anyways.
Igal Shilman [Wed, 1 Sep 2021 11:32:23 +0000 (13:32 +0200)]
[FLINK-23964] Use shaded tcnative in dev docker image
sjwiesman [Wed, 1 Sep 2021 14:03:07 +0000 (09:03 -0500)]
[hotfix] link to golang sdk from homepage
sjwiesman [Tue, 31 Aug 2021 16:54:43 +0000 (11:54 -0500)]
[hotfix][docs] remove trailing seperator
Tzu-Li (Gordon) Tai [Fri, 27 Aug 2021 06:11:11 +0000 (14:11 +0800)]
[FLINK-23600] [docs] Restructure module configuration docs
This closes #263
Stephan Ewen [Thu, 26 Aug 2021 11:11:07 +0000 (13:11 +0200)]
[hotfix] Add Java 11 profile to make StateFun work in IDEs (IntelliJ) that use JDK 11+
Stephan Ewen [Thu, 26 Aug 2021 11:05:56 +0000 (13:05 +0200)]
[hotfix][build] Add version for build helper plugin in statefun-protobuf-shaded
This commit pins the version to what was effectively used by the build process.
sjwiesman [Mon, 23 Aug 2021 20:15:58 +0000 (15:15 -0500)]
[hotfix] document go mod requirement
This closes #260.
sjwiesman [Mon, 23 Aug 2021 17:14:55 +0000 (12:14 -0500)]
[FLINK-23927] Fix go.mod file for SDK release
Tzu-Li (Gordon) Tai [Mon, 23 Aug 2021 13:36:05 +0000 (21:36 +0800)]
[hotfix] Add v1 to OkHttp transport's typename
Igal Shilman [Mon, 23 Aug 2021 06:42:22 +0000 (08:42 +0200)]
[release] Update version to 3.2-SNAPSHOT
Tzu-Li (Gordon) Tai [Wed, 18 Aug 2021 07:55:49 +0000 (15:55 +0800)]
[FLINK-23853] Update Flink dependency to v1.13.2
This closes #258.
sjwiesman [Thu, 19 Aug 2021 15:06:53 +0000 (10:06 -0500)]
[FLINK-23878][go] make cell Write resumable
This closes #259
sjwiesman [Wed, 18 Aug 2021 13:34:58 +0000 (08:34 -0500)]
[hotfix][go] harden NewCell creation checks
sjwiesman [Wed, 18 Aug 2021 13:24:56 +0000 (08:24 -0500)]
[hotfix] add smoke test checkpoint directories to .gitignore
sjwiesman [Tue, 17 Aug 2021 20:10:37 +0000 (15:10 -0500)]
[hotfix] fix typo in log statement
sjwiesman [Wed, 11 Aug 2021 14:19:36 +0000 (09:19 -0500)]
[FLINK-18810][sdk] Document Golang SDK
This closes #250
sjwiesman [Wed, 11 Aug 2021 14:18:19 +0000 (09:18 -0500)]
[FLINK-18810][sdk] Golang remote functions smoke test
sjwiesman [Wed, 11 Aug 2021 14:14:31 +0000 (09:14 -0500)]
[FLINK-18810][sdk] Golang remote functions SDK
Tzu-Li (Gordon) Tai [Sun, 15 Aug 2021 01:34:46 +0000 (09:34 +0800)]
[FLINK-22752] More robust state configurations in base flink-conf.yaml
This closes #257.
Tzu-Li (Gordon) Tai [Mon, 16 Aug 2021 06:20:03 +0000 (14:20 +0800)]
[hotfix] [e2e, build] Move driver module copying to prepare-package phase
This was causing the doc builds to fail with "Artifact has not been
packaged yet. When used on reactor artifact, copy should be executed
after packaging: see MDEP-187.
According to MDEP-187, moving the copy to prepare-package solves the
issue.
Igal Shilman [Tue, 10 Aug 2021 15:39:59 +0000 (17:39 +0200)]
[FLINK-23711] Enable async transport for the remote java-sdk e2e smoke test.
This closes #252.
Igal Shilman [Tue, 10 Aug 2021 15:37:19 +0000 (17:37 +0200)]
[FLINK-23711] Add an asynchronous HTTP transport
This commits adds a new transport based on a shaded version of Netty.
This transport is currently opt-in, and can be enabled by adding to
the endpoint definition:
...
transport:
type: io.statefun.transports.v1/async
Tzu-Li (Gordon) Tai [Thu, 12 Aug 2021 04:13:26 +0000 (12:13 +0800)]
[FLINK-20574] [core] Block first request to mitigate excessive IncompleteInvocationContext roundtrips
For the first request, we block until response is received; for stateful
applications, especially at restore time of a restored execution where
there may be a large backlog of events and checkpointed inflight
requests, this helps mitigate excessive hoards of
IncompleteInvocationContext responses and retry attempt round-trips.
This closes #256.
Tzu-Li (Gordon) Tai [Tue, 10 Aug 2021 12:37:48 +0000 (20:37 +0800)]
[FLINK-23709] Remove SanityVerificationE2E and ExactlyOnceRemoteE2E
This closes #249.
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 04:48:05 +0000 (12:48 +0800)]
[FLINK-23718] [build, sdk] Manage Python SDK as a Maven module
This closes #255.
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 04:37:41 +0000 (12:37 +0800)]
[FLINK-23717] Allow setting configs using plain strings in StatefulFunctionsAppContainers
This closes #254.
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 03:53:02 +0000 (11:53 +0800)]
[FLINK-23714] Expose both master and worker logs of StatefulFunctionsAppContainers
Igal Shilman [Tue, 10 Aug 2021 16:13:19 +0000 (18:13 +0200)]
[FLINK-23712] Improve Protobuf serialization performance
Profiling allocations shows that around 25% of the allocations happens
while parsing Protobuf, this happens because
the CodedInputStream/CodedOutputStream allocate a 4kb buffer on construction.
This can be easily avoided by caching and reusing them.
This closes #251.
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 12:17:35 +0000 (20:17 +0800)]
[hotfix] Incorrect toString() message in SmokeRunnerParameters
sjwiesman [Tue, 10 Aug 2021 21:51:07 +0000 (16:51 -0500)]
[hotfix] Add configuration for reproducible smoke tests
This closes #253.
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 10:19:52 +0000 (18:19 +0800)]
[hotfix] [e2e] Simplify target function id creation in Java smoke E2E
Tzu-Li (Gordon) Tai [Wed, 11 Aug 2021 10:09:16 +0000 (18:09 +0800)]
[hotfix] [e2e] Pull smoke E2E test utilities and Commands into a common module
Tzu-Li (Gordon) Tai [Mon, 9 Aug 2021 13:33:37 +0000 (21:33 +0800)]
[hotfix] Let only *ITCase tests be picked up by surefire
Tzu-Li (Gordon) Tai [Mon, 9 Aug 2021 12:57:11 +0000 (20:57 +0800)]
[hotfix] Recreate shared OkHttpClient after shutdown in DefaultHttpRequestReplyClientFactory
Tzu-Li (Gordon) Tai [Tue, 3 Aug 2021 07:16:16 +0000 (15:16 +0800)]
[FLINK-23600] Rework remote module component parsing and binding
This closes #247.
Evans Ye [Fri, 6 Aug 2021 18:37:58 +0000 (02:37 +0800)]
[FLINK-23672] Add smoke-e2e-java for Java SDK smoke test
This closes #248.
sjwiesman [Thu, 29 Jul 2021 15:47:33 +0000 (10:47 -0500)]
[FLINK-23547][build] run integration tests on install
This closes #246
sjwiesman [Tue, 27 Jul 2021 17:06:14 +0000 (12:06 -0500)]
[hotfix][docs] fix typos in documentation
Tzu-Li (Gordon) Tai [Fri, 23 Jul 2021 07:23:50 +0000 (15:23 +0800)]
[docs] Change base urls to https://nightlies.apache.org
Tzu-Li (Gordon) Tai [Wed, 7 Jul 2021 07:43:59 +0000 (15:43 +0800)]
[FLINK-23296] [core] Introduce RequestReplyClientFactory extension
* Rename default HTTP request-reply client
* Implement default HTTP client factory
* Wire in RequestReplyClientFactory into HttpFunctionEndpointProvider
* Pass in ExtensionResolver when binding JsonEntities
* Bump YAML format version
* Implement v3.1 YAML format with transport spec parsing
* Implement context-safe decorator for RequestReplyClient
* Ensure correct classloader when using tranport clients and factories
* Move Classloader handling responsibility to factory implementations
* Add TransportClientsModule
This closes #243.
Konstantin Knauf [Tue, 25 May 2021 13:06:42 +0000 (15:06 +0200)]
[FLINK-22771][java-sdk] Add testing utilities
* add hashCode/equals to EgressMessageWrapper/MessageWrapper
* add TestContext to Java SDK
* add documentation for Java SDK TestContext
* address review comments
* Fix various code style warnings
* Refactor Envelopes -> SideEffects
* ValueSpec needs to be identified by name, as it does not support equality.
* Support message cancellation
This closes #236.
Evans Ye [Tue, 13 Jul 2021 03:25:38 +0000 (11:25 +0800)]
[FLINK-23126] Refactor smoke-e2e into smoke-e2e-common and smoke-e2e-embedded
This closes #242.
Igal Shilman [Mon, 14 Jun 2021 10:45:10 +0000 (12:45 +0200)]
[FLINK-21308][core] Support delayed message cancellation
This closes #241.
austin ce [Wed, 16 Jun 2021 19:45:26 +0000 (15:45 -0400)]
[hotfix] Correct endpoint definition in docs
This closes #240.
Tzu-Li (Gordon) Tai [Fri, 28 May 2021 07:33:17 +0000 (15:33 +0800)]
[FLINK-22529][kinesis] Allow Flink's ConsumerConfigConstants and flexibility in providing AWS region and credentials
This closes #237.
Tzu-Li (Gordon) Tai [Tue, 25 May 2021 07:41:57 +0000 (15:41 +0800)]
[FLINK-20336] [core] Do not silently pass UNRECOGNIZED state mutations
This closes #235.
Tzu-Li (Gordon) Tai [Mon, 26 Apr 2021 05:27:15 +0000 (13:27 +0800)]
[FLINK-22468] Add dependency to javax.annotations
This enables building StateFun with Java 11, since javax.annotations was
removed with JDK 11.
This closes #230.
Seth Wiesman [Mon, 12 Apr 2021 16:32:01 +0000 (11:32 -0500)]
[FLINK-22242][python] Improve missing attribute error on GeneratedAddressedScopedStorage
This closes #223
Tzu-Li (Gordon) Tai [Tue, 18 May 2021 06:15:03 +0000 (14:15 +0800)]
[FLINK-22552] Update Flink dependency to 1.13.0
This closes #234.
Seth Wiesman [Tue, 11 May 2021 15:34:14 +0000 (10:34 -0500)]
[hotfix] Add duration to YAML config types
Seth Wiesman [Tue, 4 May 2021 18:55:52 +0000 (13:55 -0500)]
[hotfix] Cross link ingress and egress module configuration to io module section
Seth Wiesman [Tue, 4 May 2021 18:04:53 +0000 (13:04 -0500)]
[hotfix][docs] fix invalid module configuration
This closes #232
ariskk [Sat, 24 Apr 2021 10:31:08 +0000 (11:31 +0100)]
[hotfix][docs] Fix typos in concepts
This closes #229.
ariskk [Sat, 24 Apr 2021 10:04:14 +0000 (11:04 +0100)]
[hotfix][docs] Fix typos in apache-kafka.md and aws-kinesis.md
Seth Wiesman [Thu, 22 Apr 2021 16:44:54 +0000 (11:44 -0500)]
[hotfix] Add missing configuration to documentation