Release 1.8.1 of the Amazon Kinesis Client (#198)

Support timeouts for calls to the MultiLang Daemon
This adds support for setting a timeout when dispatching records to
the client record processor. If the record processor doesn't respond
within the timeout the parent Java process will be terminated. This
is a temporary fix to handle cases where the KCL becomes blocked
while waiting for a client record processor.

The timeout for the this can be set by adding
  `timeoutInSeconds = <timeout value>`.
The default for this is no timeout.

Setting this can cause the KCL to exit suddenly, before using this
ensure that you have an automated restart for your application

Related #195
Related #185
This commit is contained in:
Justin Pfifer 2017-08-02 10:41:14 -07:00 committed by GitHub
parent 526a1fa0df
commit 79296dad2e
4 changed files with 13 additions and 5 deletions

View file

@ -2,7 +2,7 @@ Manifest-Version: 1.0
Bundle-ManifestVersion: 2 Bundle-ManifestVersion: 2
Bundle-Name: Amazon Kinesis Client Library for Java Bundle-Name: Amazon Kinesis Client Library for Java
Bundle-SymbolicName: com.amazonaws.kinesisclientlibrary;singleton:=true Bundle-SymbolicName: com.amazonaws.kinesisclientlibrary;singleton:=true
Bundle-Version: 1.8.0 Bundle-Version: 1.8.1
Bundle-Vendor: Amazon Technologies, Inc Bundle-Vendor: Amazon Technologies, Inc
Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Require-Bundle: org.apache.commons.codec;bundle-version="1.6", Require-Bundle: org.apache.commons.codec;bundle-version="1.6",

View file

@ -23,12 +23,20 @@ The **Amazon Kinesis Client Library for Java** (Amazon KCL) enables Java develop
After you've downloaded the code from GitHub, you can build it using Maven. To disable GPG signing in the build, use this command: `mvn clean install -Dgpg.skip=true` After you've downloaded the code from GitHub, you can build it using Maven. To disable GPG signing in the build, use this command: `mvn clean install -Dgpg.skip=true`
## Integration with the Kinesis Producer Library ## Integration with the Kinesis Producer Library
For producer-side developers using the **[Kinesis Producer Library (KPL)][kinesis-guide-kpl]**, the KCL integrates without additional effort.  When the KCL retrieves an aggregated Amazon Kinesis record consisting of multiple KPL user records, it will automatically invoke the KPL to extract the individual user records before returning them to the user. For producer-side developers using the **[Kinesis Producer Library (KPL)][kinesis-guide-kpl]**, the KCL integrates without additional effort. When the KCL retrieves an aggregated Amazon Kinesis record consisting of multiple KPL user records, it will automatically invoke the KPL to extract the individual user records before returning them to the user.
## Amazon KCL support for other languages ## Amazon KCL support for other languages
To make it easier for developers to write record processors in other languages, we have implemented a Java based daemon, called MultiLangDaemon that does all the heavy lifting. Our approach has the daemon spawn a sub-process, which in turn runs the record processor, which can be written in any language. The MultiLangDaemon process and the record processor sub-process communicate with each other over [STDIN and STDOUT using a defined protocol][multi-lang-protocol]. There will be a one to one correspondence amongst record processors, child processes, and shards. For Python developers specifically, we have abstracted these implementation details away and [expose an interface][kclpy] that enables you to focus on writing record processing logic in Python. This approach enables KCL to be language agnostic, while providing identical features and similar parallel processing model across all languages. To make it easier for developers to write record processors in other languages, we have implemented a Java based daemon, called MultiLangDaemon that does all the heavy lifting. Our approach has the daemon spawn a sub-process, which in turn runs the record processor, which can be written in any language. The MultiLangDaemon process and the record processor sub-process communicate with each other over [STDIN and STDOUT using a defined protocol][multi-lang-protocol]. There will be a one to one correspondence amongst record processors, child processes, and shards. For Python developers specifically, we have abstracted these implementation details away and [expose an interface][kclpy] that enables you to focus on writing record processing logic in Python. This approach enables KCL to be language agnostic, while providing identical features and similar parallel processing model across all languages.
## Release Notes ## Release Notes
### Release 1.8.1 (August 2, 2017)
* Support timeouts for calls to the MultiLang Daemon
This adds support for setting a timeout when dispatching records to the client record processor. If the record processor doesn't respond within the timeout the parent Java process will be terminated. This is a temporary fix to handle cases where the KCL becomes blocked while waiting for a client record processor.
The timeout for the this can be set by adding `timeoutInSeconds = <timeout value>`. The default for this is no timeout.
__Setting this can cause the KCL to exit suddenly, before using this ensure that you have an automated restart for your application__
* [PR #195](https://github.com/awslabs/amazon-kinesis-client/pull/195)
* [Issue #185](https://github.com/awslabs/amazon-kinesis-client/issues/185)
### Release 1.8.0 (July 25, 2017) ### Release 1.8.0 (July 25, 2017)
* Execute graceful shutdown on its own thread * Execute graceful shutdown on its own thread
* [PR #191](https://github.com/awslabs/amazon-kinesis-client/pull/191) * [PR #191](https://github.com/awslabs/amazon-kinesis-client/pull/191)
@ -36,7 +44,7 @@ To make it easier for developers to write record processors in other languages,
* Added support for controlling the size of the lease renewer thread pool * Added support for controlling the size of the lease renewer thread pool
* [PR #177](https://github.com/awslabs/amazon-kinesis-client/pull/177) * [PR #177](https://github.com/awslabs/amazon-kinesis-client/pull/177)
* [Issue #171](https://github.com/awslabs/amazon-kinesis-client/issues/171) * [Issue #171](https://github.com/awslabs/amazon-kinesis-client/issues/171)
* Require Java 8 and later * Require Java 8 and later
__Java 8 is now required for versions 1.8.0 of the amazon-kinesis-client and later.__ __Java 8 is now required for versions 1.8.0 of the amazon-kinesis-client and later.__
* [PR #176](https://github.com/awslabs/amazon-kinesis-client/issues/176) * [PR #176](https://github.com/awslabs/amazon-kinesis-client/issues/176)

View file

@ -6,7 +6,7 @@
<artifactId>amazon-kinesis-client</artifactId> <artifactId>amazon-kinesis-client</artifactId>
<packaging>jar</packaging> <packaging>jar</packaging>
<name>Amazon Kinesis Client Library for Java</name> <name>Amazon Kinesis Client Library for Java</name>
<version>1.8.1-SNAPSHOT</version> <version>1.8.1</version>
<description>The Amazon Kinesis Client Library for Java enables Java developers to easily consume and process data <description>The Amazon Kinesis Client Library for Java enables Java developers to easily consume and process data
from Amazon Kinesis. from Amazon Kinesis.
</description> </description>

View file

@ -126,7 +126,7 @@ public class KinesisClientLibConfiguration {
/** /**
* User agent set when Amazon Kinesis Client Library makes AWS requests. * User agent set when Amazon Kinesis Client Library makes AWS requests.
*/ */
public static final String KINESIS_CLIENT_LIB_USER_AGENT = "amazon-kinesis-client-library-java-1.8.0"; public static final String KINESIS_CLIENT_LIB_USER_AGENT = "amazon-kinesis-client-library-java-1.8.1";
/** /**
* KCL will validate client provided sequence numbers with a call to Amazon Kinesis before checkpointing for calls * KCL will validate client provided sequence numbers with a call to Amazon Kinesis before checkpointing for calls