* Revalidate if current shard is closed before shutting down the ShardConsumer
* KCL 2.2.5 release
* KCL 2.2.5 release
* Fixing bad merge
* Update the KINESIS_CLIENT_LIB_USER_AGENT_VERSION
* Revalidate if current shard is closed before shutting down the ShardConsumer
* Renaming Method
* Force Lease to be lost before shutting down with Zombi state
* Adding comments for ShardEnd related unit tests
* Making test cases resilient to delayed thread operations
* Setting the initial demand in test cases to be in line with service's coral initial demand.
* Fix to prevent the onNext event going to stale subscription when restart happens in poller
* Isolating session variables into a new class. Replacing thread control shifting logic for publishing with monitor based control
* Refactoring based on review comments
* Addressing review comments on unit test cases
* Preventing duplicate delivery due to unacknowledged event while completing the subscription
* Refactored clearRecordsDeliveryQueue logic and added comments
* Code refactoring as per review comments
* Nit fix
* Add logging to unexpected subscription state scenario
* Adding unit test case for record delivery validation
* Initial prototype for notification mechanism between ShardConsumerSubscriber and FanoutPublisher. The SDK Threads are made to block wait on the ack from the ShardConsumerSubscriber
* initial non blocking prototype
* Refactoring src and test
* Added unit test cases. Addressed review comments. Handled edge cases
* Minor code changes. Note that the previous commit has blocking impl of PrefetchPublisher
* Refactored the cleanup logic
* Fix for Cloudwatch exception handling and other revioew comment fixes
* Typo fix
* Removing cloudwatch fix. Will be released in a separate commit.
* Changing RejectedTaskEvent log message for the release
* Added javadoc to RecordsDeliveryAck and optimized imports
* Adding Kinesis Internal API tag for new concrete implementations
* Making CW publish calls as blocking to reduce the throttling. Disclosing the CW publish failures.
* Fixing uniut test cases and adding CW exception manager
* Fixing bug where initial subscription fails cause shard consumer to get stuck.
* Adding some comments for the changes and simplifying the unit test.
* Adding unit tests for handling restart in case of rejection execution exception from executor service.
* Fix to prevent ShardConsumer state transition, when the source state task execution is rejected by the executor service.
* Unit test case improvements
* Optimized imports
* Removed unnecessary sleep in unit test case
* Fixing imports
* Fixing import again with wildcard removed
* Adding asserts to exception cases in SharConsumerTest
* Add diagnostic events for logging visibility
* Refactor logging diagnostics into main Scheduler loop
* Refactor log timing and level and change privacies
* Revert ExecutorStateEvent to accept ExecutorService input type
* Minor style and messaging fixes
* Fix failing unit test
* Refactor diagnostic events to use factory for testing
* Fix constructor overloading for testing
* Refactor DiagnosticEventHandler to no args constructor
* Added configuration to ignore a number of ReadTimeouts before printing warnings.
Messaging now directs customer to configure the SDK with appropriate timeouts based on their processing model.
Warning messages from ShardConsumer now specify that the KCL will reattempt to subscribe to the stream as needed.
Added configuraiton to Lifecycle configuration to enable ignoring a number of ReadTimeouts before printing warning messages.
* Removed functional tests that are now replicated as unit tests
* Refactored after review of Pull Request
Marked original ShardConsumerSubscriber constructor as deprecated
Renamed tests to be more descriptive
* Updated Default Value injection to ShardConsumerSubscriber
* Refactored based on PR comments
* Removed Chained Constructor from test class
* Added comments to tests to make then easier to understand
* Seperating coding of each log suppression test
* Advance version to 2.1.3-SNAPSHOT
* Added timeouts for Kinesis and DynamoDB calls
Added a timeout to prevent an issue where the Kinesis or DynamoDB call
never completes.
For Kinesis call to GetRecords the timeout defaults to 30 seconds, and can be configured
on the PollingConfig.
For DynamoDB and Kinesis (when calling ListShards) the timeout
defaults to 60 seconds and can be configured on LeaseManagementConfig.
The KinesisDataFetcher uses the AWSExceptionManager to translate
execution exceptions into the expected exceptions. Currently if the
exception is unexpected the exception will be wrapped in a
RuntimeException before being returned. We depend on SdkExceptions
being the right type where we handle them upstream so we add a
configuration for SdkException which should ensure handling works as
expected.
If the Scheduler loses its lease for a shard it will attempt to
shutdown the ShardConsumer processing that shard. When shutting down
the ShardConsumer acquires a lock on `this` and makes the necessary
state changes.
This becomes an issue if the ShardConsumer is currently processing a
batch of records as processing of the records is done under the
general `this` lock.
When these two things combine the Scheduler can become stuck waiting
on the record processing to complete.
To fix this the ShardConsumer will now use a specific lock on shutdown
state changes to prevent the Scheduler from becoming blocked.
Allow the shutdown state change future to acquire the lock
When the ShardConsumer is being shutdown we create a future for the
state change originally the future needed to acquire the lock before
attempting to create the future task. This changes it to acquire the
lock while running on another thread, and complete the shutdown then.
