* Renamed the retrieval strategy classes to fix a spelling error.
* Modified the strategy interface to support shutdown, and determination
of whether a strategy has been shutdown.
* Moved the existing tests for the async strategy to an integration
test.
* Modified the async strategy to allow injection of more state
components
* Modified the async strategy to throw an exception if an attempt is
made to use it after shutdown.
cr https://code.amazon.com/reviews/CR-590341
Adds support for making two phase commits. When a checkpoint is started, but fails to complete the next instance of the record processor will receive the attempted checkpoint position at initialization time.
Support timeouts for calls to the MultiLang Daemon
This adds support for setting a timeout when dispatching records to
the client record processor. If the record processor doesn't respond
within the timeout the parent Java process will be terminated. This
is a temporary fix to handle cases where the KCL becomes blocked
while waiting for a client record processor.
The timeout for the this can be set by adding
`timeoutInSeconds = <timeout value>`.
The default for this is no timeout.
Setting this can cause the KCL to exit suddenly, before using this
ensure that you have an automated restart for your application
Related #195
Related #185
* Adding timeout to waitForStatusMessage future call. Introducing new config properties timeoutEnabled and timeoutInSeconds. Halting the JVM if timeout is reached.
* Adding test cases for halt jvm code. Made the configuration objects for timeout optional.
* Addressing code review comments and making appropriate changes.
* Execute graceful shutdown on its own thread
* PR #191
* Issue #167
* Added support for controlling the size of the lease renewer thread pool
* PR #177
* Issue #171
* Require Java 8 and later
Java 8 is now required for versions 1.8.0 of the amazon-kinesis-client and later.
* PR #176
* Initial start of fix for requested shutdown
* Execute the requested shutdown in a separate thread.
Fix for Issue #167
* Reworked some of the shutdown logic to make the relationships clearer.
* Added/Updated Copyright Statements
* Add Missing License Statements
Allow configuration of the lease renewer thread pool size. Users can
now control the size of the pool. Additionally core threads are now
a quarter of the thread pool size, and aren't allowed to timeout.
* Advance version, and drop Java 7 support
Advanced the version to 1.8.0 as Java 7 support is being removed.
* Remove build settings for Java 7
Removed the configuration for Java 7, and switched to trusty for access
to openjdk8.
* Added support for graceful shutdown in MultiLang Clients
* PR #174
* PR #182
* Updated documentation for `v2.IRecordProcessor#shutdown`, and `KinesisClientLibConfiguration#idleTimeBetweenReadsMillis`
* PR #170
* Updated to version 1.11.151 of the AWS Java SDK
* PR #183
When the parent Java process receives a SIGTERM it will now trigger a graceful shutdown of the worker, which dispatches a ShutdownRequestedMessage to all of the record processors. This will allow record processors a final chance to checkpoint before the lease is terminated.
This changes is incompatible with current versions of the MultiLang Clients. A future change will allow older versions MultiLang clients to use newer versions of the Java KCL.
* Correctly handle throttling for DescribeStream, and save accumulated progress from individual calls.
* PR #152
* Upgrade to version 1.11.115 of the AWS Java SDK
* PR #155
Improve the handling of describe stream throttling by no longer
triggering a null pointer exception when all requests are throttled.
Also store the last position reached, and always restart from
there.
When the dynamoDBEndpoint configuration property was added, not all paths that
should have made use of it were updated. Both paths now use the config value to
set up the DynamoDB client.
The existing ThreadPoolExecutor was misconfigured and caused
the thread pool to recycle idle threads continuously. VisualVM
showed that a thread got recycled about every 3 seconds, creating
and destroying several hundred threads in about 15 minutes.
Use ExecutorService.newFixedThreadPool instead, as recommended
by the javadoc for ExecutorService.
Disable the JavaDoc doclint when building with Java 8. Java 8 became
much more pedantic on bad practices, and the documentation hasn't been
cleaned up.
* Reduce Throttling Error Reports
Only report a throttling error if their are 6 consecutive throttles.
Moved the logging of throttling messages to the throttling
reporter.
* Handle No Lease, and Lease Loss in requestShutdown.
Ensure that the Worker is shutdown if there are no leases. Handle the
case where a lease is lost during the notification process.
* Add some more comments around possible race conditions.
Made the ShardInfo class public, and updated the documentation for a
number of the properties.
This will allow users to implement custom shard prioritization
strategies.
Fixes:
Issue #120
Updated to the newest version of the AWS Java SDK.
Fixes:
Amazon Kinesis Client for Python Issue awslabs/amazon-kinesis-client-python#27
Related:
PR #125
* Add Direct Dependency on Commons Log
Commons log was a transient dependency, but should have been a direct
dependency. Now take a direct dependency on it.
* Move to Next Release Version Snapshot
Changed version to 1.7.3-SNAPSHOT for next release.
MultiLangDaemon Feature Updates
The MultiLangDaemon has been upgraded to use the v2 interfaces, which allows access to enhanced checkpointing, and more information during record processor initialization. The MultiLangDaemon clients must be updated before they can take advantage of these new features.
