Commit graph

587 commits

Author SHA1 Message Date
Micah Jaffe
ae0d334508
Prepare KCL 2.2.7 release (#658) 2019-12-02 13:33:29 -08:00
ychunxue
5ac2752a91 Upgrade SDK to version 2.10.25 (#657) 2019-12-02 13:25:47 -08:00
Eric Meisel
8b7c1554cb Configurable DynamoDB BillingMode (#582)
* Configurable DDB BillingModes for LeaseTables upon creation

* Adding tests and fixing specifying DDB provisioning levels when billing by request.
2019-11-26 12:32:56 -08:00
ychunxue
80f5f68765
Merge pull request #650 from ychunxue/master
Prepare for KCL release 2.2.6
2019-11-07 17:18:13 -08:00
Chunxue Yang
b7775e7cf1 Update Release Note 2019-11-07 17:15:03 -08:00
Chunxue Yang
b1ca4f860e Fix unit tests 2019-11-07 16:49:15 -08:00
Chunxue Yang
5221e1cd54 Fixing unit tests 2019-11-07 15:32:33 -08:00
Chunxue Yang
d940f8e2aa Potential Unit test fix 2019-11-07 13:52:00 -08:00
Chunxue Yang
653a25456c Debug remote unit test failure 2019-11-07 13:37:45 -08:00
Chunxue Yang
6690812aac Prepare for KCL release 2.2.6 2019-11-06 15:26:32 -08:00
ychunxue
ff8794f05b
Merge pull request #649 from ychunxue/master
Upgrade SDK to 2.10.0
2019-11-06 11:49:36 -08:00
Chunxue Yang
d5dca1a91e Upgrade SDK to 2.10.0 2019-11-06 10:51:47 -08:00
ychunxue
047493aa9e
Merge pull request #642 from ychunxue/master
Remove the shutdown event from the queue before executing the shudown…
2019-11-01 15:45:32 -07:00
Chunxue Yang
5239ed96ff Addressing comments 2019-11-01 14:08:57 -07:00
Chunxue Yang
c5c4e428f2 Remove the shutdown event from the queue before executing the shudown event 2019-11-01 09:47:24 -07:00
ychunxue
c696bc1862
Merge pull request #638 from ychunxue/master
Updating sdk version and fixing unit tests
2019-10-30 14:05:03 -07:00
Chunxue Yang
c7754c4eda Fixing unit tests 2019-10-29 11:57:17 -07:00
Chunxue Yang
079bb52611 Updating sdk version and fixing unit tests 2019-10-25 15:08:20 -07:00
ychunxue
6dc25fbc99
Merge pull request #636 from ychunxue/master
SNAPSHOT version create
2019-10-24 14:31:44 -07:00
Chunxue Yang
6d3b042f6d SNAPSHOT version create 2019-10-24 14:09:27 -07:00
ychunxue
bb8fa6cd89 KCL Release 2.2.5 (#635)
* Revalidate if current shard is closed before shutting down the ShardConsumer

* KCL 2.2.5 release

* KCL 2.2.5 release

* Fixing bad merge

* Update the KINESIS_CLIENT_LIB_USER_AGENT_VERSION
2019-10-23 15:54:23 -07:00
ychunxue
1f686488c7 Shard end v2 (#624)
* Revalidate if current shard is closed before shutting down the ShardConsumer

* Renaming Method

* Force Lease to be lost before shutting down with Zombi state

* Adding comments for ShardEnd related unit tests
2019-10-23 09:33:07 -07:00
Micah Jaffe
6fa9bab66e
Update Sonatype to use dedicated AWS endpoint (#619) 2019-10-04 13:53:14 -07:00
Micah Jaffe
98b016276b
Update README.md 2019-09-23 14:46:42 -07:00
ashwing
0ca5b4d6f6 KCL 2.2.4 release (#615) 2019-09-23 07:51:12 -07:00
ashwing
c197d35aab Making test cases resilient to delayed thread operations (#612)
* Making test cases resilient to delayed thread operations

* Setting the initial demand in test cases to be in line with service's coral initial demand.
2019-09-19 16:53:01 -07:00
ashwing
3fead19df7 Fix to prevent the onNext event going to stale subscription when restart happens in poller (#606)
* Fix to prevent the onNext event going to stale subscription when restart happens in poller

* Isolating session variables into a new class. Replacing thread control shifting logic for publishing with monitor based control

* Refactoring based on review comments

* Addressing review comments on unit test cases
2019-09-19 10:04:20 -07:00
ashwing
a94dc7d61d Drain delivery queue to make slow consumers consume events at their pace (#607)
* Allowing consumers to drain the delivery queue on subscription end

* Test cases fix

* Added test cases

* Made feedback changes
2019-09-13 17:01:04 -07:00
Micah Jaffe
db94cb60ef
Update SNAPSHOT version to 2.2.4-SNAPSHOT (#605) 2019-09-04 16:37:07 -07:00
ashwing
18a55cf6f3 KCL 2.2.3 release (#604)
* KCL 2.2.3 release

* Added prior related release notes
2019-09-04 12:17:20 -07:00
ashwing
f6dec3e579 Fix to prevent data loss and stuck shards in the event of failed records delivery in Polling readers (#603)
* Fix to prevent data loss and stuck shards in the event of failed records delivery.

* Review comment fixes

* Access specifiers fix
2019-09-03 09:20:34 -07:00
Cory-Bradshaw
85d31c91f1
Merge pull request #599 from ashwing/v2.2.3-snapshot
Creating snapshot version for 2.2.3 release
2019-08-20 12:01:59 -07:00
Ashwin Giridharan
20cce175be Creating snapshot version for 2.2.3 release 2019-08-20 11:52:25 -07:00
ashwing
b8331f76e3 KCL 2.2.2 release (#598)
* KCL 2.2.2 release

* Addressing release notes feedback
2019-08-19 16:27:34 -07:00
ashwing
a17d14527a Preventing duplicate delivery due to unacknowledged event while completing the subscription (#596)
* Preventing duplicate delivery due to unacknowledged event while completing the subscription

* Refactored clearRecordsDeliveryQueue logic and added comments

* Code refactoring as per review comments

* Nit fix

* Add logging to unexpected subscription state scenario
2019-08-19 14:35:06 -07:00
ashwing
3f6afc6563 Limited threads resiliency fix durability nonblock (#573)
* Adding unit test case for record delivery validation

* Initial prototype for notification mechanism between ShardConsumerSubscriber and FanoutPublisher. The SDK Threads are made to block wait on the ack from the ShardConsumerSubscriber

* initial non blocking prototype

* Refactoring src and test

* Added unit test cases. Addressed review comments. Handled edge cases

* Minor code changes. Note that the previous commit has blocking impl of PrefetchPublisher

* Refactored the cleanup logic

* Fix for Cloudwatch exception handling and other revioew comment fixes

* Typo fix

* Removing cloudwatch fix. Will be released in a separate commit.

* Changing RejectedTaskEvent log message for the release

* Added javadoc to RecordsDeliveryAck and optimized imports

* Adding Kinesis Internal API tag for new concrete implementations
2019-08-16 14:24:19 -07:00
Micah Jaffe
c2a3f18670 Update ShardEnd checkpoint failure messaging (#591)
* Update shard end checkpoint failure messaging

* Update shard end checkpoint failure messaging
2019-08-13 13:18:52 -07:00
ashwing
161590c2ce Adding wait to CW PutMetric future calls (#584)
* Making CW publish calls as blocking to reduce the throttling. Disclosing the CW publish failures.

* Fixing uniut test cases and adding CW exception manager
2019-08-09 09:52:53 -07:00
yatins47
a150402e9c Fixing bug where initial subscription failure causes shard consumer to get stuck. (#562)
* Fixing bug where initial subscription fails cause shard consumer to get stuck.

* Adding some comments for the changes and simplifying the unit test.

* Adding unit tests for handling restart in case of rejection execution exception from executor service.
2019-07-11 07:16:04 -07:00
ashwing
9e2d6fa497 Fix for invalid ShardConsumer state transitions due to rejected executions (#560)
* Fix to prevent ShardConsumer state transition, when the source state task execution is rejected by the executor service.

* Unit test case improvements

* Optimized imports

* Removed unnecessary sleep in unit test case

* Fixing imports

* Fixing import again with wildcard removed

* Adding asserts to exception cases in SharConsumerTest
2019-07-08 16:30:27 -07:00
Micah Jaffe
b6236d8077 Update version to 2.2.2-SNAPSHOT (#565) 2019-07-05 12:49:53 -07:00
Micah Jaffe
7d8b281d24
Prepare for v2.2.1 (#561)
* Prepare for v2.2.1

* Minor messaging fixes

* Update CHANGELOG.md

* Update README.md

* Update CHANGELOG.md

* Update README.md

* Update CHANGELOG.md

* Update README.md
2019-07-01 11:49:56 -07:00
Micah Jaffe
fa72cf1517 Adding more logging around the rejected task executions at the Scheduler and RxJava layer (#559)
* Add diagnostic events for logging visibility

* Refactor logging diagnostics into main Scheduler loop

* Refactor log timing and level and change privacies

* Revert ExecutorStateEvent to accept ExecutorService input type

* Minor style and messaging fixes

* Fix failing unit test

* Refactor diagnostic events to use factory for testing

* Fix constructor overloading for testing

* Refactor DiagnosticEventHandler to no args constructor
2019-06-28 10:02:57 -07:00
Cory-Bradshaw
6159b869ed 2.2.1-SNAPSHOT Updated Snapshot versioning (#544)
* 2.2.1-SNAPSHOT Updated Snapshot versioning

* Updated 1.x versioning in README
2019-04-09 12:15:33 -07:00
Cory-Bradshaw
9a15971bde Added @RunWith to integration tests for LeaseIntegrationTests (#543) 2019-04-09 10:00:38 -07:00
Cory-Bradshaw
c8f82836b1 Preparation for v2.2.0 (#536) 2019-04-08 11:20:08 -07:00
awslankakamal
6c64055d9b Updating license to Apache License 2.0 (#523) 2019-04-05 15:25:09 -07:00
Cory-Bradshaw
2851a8b6e0 Introducing configuration for ignoring ReadTimeouts before printing warnings. (#528)
* Added configuration to ignore a number of ReadTimeouts before printing warnings.

     Messaging now directs customer to configure the SDK with appropriate timeouts based on their processing model.
     Warning messages from ShardConsumer now specify that the KCL will reattempt to subscribe to the stream as needed.
     Added configuraiton to Lifecycle configuration to enable ignoring a number of ReadTimeouts before printing warning messages.

* Removed functional tests that are now replicated as unit tests

* Refactored after review of Pull Request

Marked original ShardConsumerSubscriber constructor as deprecated
Renamed tests to be more descriptive

* Updated Default Value injection to ShardConsumerSubscriber

* Refactored based on PR comments

* Removed Chained Constructor from test class

* Added comments to tests to make then easier to understand

* Seperating coding of each log suppression test
2019-04-05 15:13:10 -07:00
Cory-Bradshaw
1bfaa90322 Updated versions to 2.1.4-SNAPSHOT (#526) 2019-03-26 11:59:10 -07:00
Justin Pfifer
a629185786 Release 2.1.3 of the Amazon Kinesis Client Library for Java (#519)
Milestone#30: https://github.com/awslabs/amazon-kinesis-client/milestone/30
* Added a message to recommend using `KinesisClientUtil` when an acquire timeout occurs in the `FanOutRecordsPublisher`.
  * PR#514: https://github.com/awslabs/amazon-kinesis-client/pull/514
* Added a sleep between retries while waiting for a newly created stream consumer to become active.
  * PR#506: https://github.com/awslabs/amazon-kinesis-client/issues/506
* Added timeouts on all futures returned from the DynamoDB and Kinesis clients.
  The timeouts can be configured by setting `LeaseManagementConfig#requestTimeout(Duration)` for DynamoDB, and `PollingConfig#kinesisRequestTimeout(Duration)` for Kinesis.
  * PR#518: https://github.com/awslabs/amazon-kinesis-client/pull/518
* Upgraded to SDK version 2.5.10.
  * PR#518: https://github.com/awslabs/amazon-kinesis-client/pull/518
* Artifacts for the Amazon Kinesis Client for Java are now signed by a new GPG key:
  pub   4096R/86368934 2019-02-14 [expires: 2020-02-14]
  uid                  Amazon Kinesis Tools <amazon-kinesis-tools@amazon.com>
2019-03-18 16:28:42 -07:00