luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
259 commits 48 branches 124 tags 11 MiB
Commit graph

259 commits

This branch
This branch
All branches
Author SHA1 Message Date
bencvdb
92ab3a610e
Merge pull request #7 from fivetran/failure-on-garbage-incomplete-leases 
fix(worker): fail on garbage incomplete leases
 2021-07-07 08:44:10 -07:00
bencvdb
17ad929831
Merge branch 'master' into failure-on-garbage-incomplete-leases 2021-07-07 08:38:04 -07:00
bencvdb
47f68875ab
Merge pull request #6 from fivetran/85045-worker-improvements 
feature(worker): missing data causes failure
 2021-07-07 08:36:14 -07:00
bencvdb
52199a24ee refactor(worker): broaden exception handling 2021-07-07 08:29:57 -07:00
bencvdb
f2dd16257d fix(worker): actually throw the error 2021-07-06 16:47:21 -07:00
bencvdb
a8551d5389 refactor(worker): consolidate exception checking logic 2021-07-06 16:40:46 -07:00
bencvdb
981bacd512 fix(worker): fail on incomplete garbage lease 2021-07-06 10:25:56 -07:00
bencvdb
c8a1797b7e fix(worker): fail on TrimmedDataAccessException (101797) 2021-07-06 10:25:56 -07:00
bencvdb
d4eb2b66e3 fix(worker): fail on TrimmedDataAccessException (101797) 2021-07-01 10:59:03 -07:00
Zekai Liu
e8dcb0a810
Merge pull request #4 from fivetran/fix_amazon_kinesis_client_check_56476 
fix(kinesis_proxy): fix AmazonKinesis client class name check
 2020-01-02 11:40:33 -08:00
zekailiu
6b287424d9 fix(kinesis_proxy): fix AmazonKinesis client class name check 2019-12-23 17:28:23 -08:00
George Fraser
124a602515
Merge pull request #3 from fivetran/shade 
Move to com.fivetran.external
 2019-12-06 07:27:26 -08:00
George Fraser
4aefa9ba1a Stray reference to old package name 2019-12-05 21:17:57 -08:00
George Fraser
f987cc7f9a Move to com.fivetran.external 2019-12-05 21:07:27 -08:00
bencvdb
88b6ff2f30
Merge pull request #2 from fivetran/throw_insufficent_permissions_error 
feature(worker): halt worker on permissions error
 2019-07-01 14:47:06 -07:00
bencvdb
a807444f57
fix(Worker): align finally 2019-07-01 14:46:35 -07:00
bencvdb
ce875f3d48 feature(worker): remove unnecessary functions and permissions 2019-06-27 09:07:17 -07:00
bencvdb
d2f95cb5ed feature(worker): catch errors from runProcessLoop and shutdown 2019-06-26 13:51:50 -07:00
bencvdb
9e102af3dd feature(worker): make Worker#getLeaseCoordinator public 2019-06-26 09:35:59 -07:00
bencvdb
08bf7f7000 feature(worker): remove RuntimeException wrapping 2019-06-25 15:01:54 -07:00
bencvdb
b8861e50b1 feature(worker): throw and handle AmazonDynamDBException 2019-06-25 14:28:02 -07:00
bencvdb
06652d8f88 feature(worker): throw insufficent permissions error 2019-06-25 11:03:14 -07:00
glarwood
4ffc50a8ff refactor(Worker): update error message 2019-03-22 18:10:29 +00:00
glarwood
8642f0b40b feat(StreamRecordProcessingError): add new error to stop worker 2019-03-22 17:58:14 +00:00
glarwood
f0230c9040 refactor(Worker): allow worker to fail after error in processing records 2019-03-22 15:17:31 +00:00
glarwood
913f2e9377 refactor(Worker): remove exits after initial lease completion code & default time limit 2019-03-20 20:07:44 +00:00
glarwood
0a4fc8bef0 refactor(ProcessTask): expose shardId 2019-03-20 18:30:39 +00:00
glarwood
0f78ff0bae fix(Worker): only save lease keys 2019-03-20 16:57:19 +00:00
glarwood
327f0722f5 refactor(Worker): exit after initial leases complete 2019-03-20 15:59:31 +00:00
Gabriel Larwood
08c45f0e42
Merge pull request #1 from fivetran/exit-when-shard-complete 
refactor(Worker): shutdown when shardInfoShardConsumerMap is empty
 2019-03-05 21:06:54 -08:00
glarwood
1a617249f4 refactor(Worker): make Fivetran specific code configurable 2019-03-05 21:04:52 -08:00
glarwood
14fb8e2703 refactor(Worker): shutdown when shardInfoShardConsumerMap is empty 2019-03-05 20:34:58 -08:00
glarwood
6fd2276556 refactor(AWSCredentialsProviderPropertyValueDecoder): cast nameAndArgs as Object[] 2018-12-13 15:04:34 -08:00
glarwood
f42037eefd fix(GracefulShutdownContext): remove non-null requirement 2018-12-13 12:12:55 -08:00
glarwood
c1b1f9b74d refactor(src): delombok the src 2018-12-13 11:57:02 -08:00
glarwood
3a91015bba refactor(Worker): throws exception after MAX_RETRIES on runProcessLoop 2018-12-12 14:27:38 -08:00
glarwood
43bbeb5fe6 revert to release 1.9.3 2018-12-12 13:58:35 -08:00
glarwood
7417524649 refactor(Worker): add retry logic on runProcessLoop. Throws exception after MAX_RETRIES 2018-12-11 19:46:40 -08:00
glarwood
a116817710 Revert back to 0a91e6faa5 2018-12-11 18:53:20 -08:00
Justin Pfifer
a05e22f782
Release 2.0.5 of the Amazon Kinesis Client for Java (#465) 2018-11-12 08:54:04 -08:00
Justin Pfifer
f52f2559ed Remove a possible deadlock on polling queue fill (#462) 
* Remove a possible deadlock on polling queue fill

Adding new items to the receive queue for the PrefetchRecordsPublisher
when at capacity would deadlock retrievals as it was already holding
a lock on this.

The method addArrivedRecordsInput did not need to be synchronized on
this as it didn't change any of the protected
state (requestedResponses).  There is a call to drainQueueForRequests
immediately after the addArrivedRecordsInput that will ensure newly
arrived data is dispatched.

This fixes #448

* Small fix on the reasoning comment

* Adjust the test to act more like the ShardConsumer

The ShardConsuemr, which is the principal user of the
PrefetchRecordsPublisher, uses RxJava to consume from publisher. This
test uses RxJava to consume, and notifies the test thread once
MAX_ITEMS * 3 have been received. This ensures that we cycle through
the queue at least 3 times.

* Removed the upper limit on the retrievals

The way RxJava's request management makes it possible that more
requests than we might expect can happen.
 2018-11-07 16:33:49 -08:00
Sahil Palvia
b83a32b492
Making configurations consistent in entire package (#453) 2018-10-25 10:40:35 -07:00
Sahil Palvia
9f9620354e Adding travis and codebuild badges to readme for v2.x branch (#456) 2018-10-25 08:00:16 -07:00
Sahil Palvia
6bd63f70dd
Updating BuildStatus badge (#452) 
* Adding new BuildStatus badges.

* Updating to show single build status
 2018-10-23 12:32:31 -07:00
Sahil Palvia
df6d25a201 Updating version to 2.0.5-SNAPSHOT (#450) 2018-10-23 08:09:59 -07:00
Sahil Palvia
eb7b3fd1bb Release 2.0.4 of the Amazon Kinesis Client for Java (#447) 
Release 2.0.4 of the Amazon Kinesis Client for Java
 2018-10-18 10:32:23 -07:00
Justin Pfifer
f2fb9ead0d
Added a sequence number validator to ensure safer checkpoints (#432) 
Added the SequenceNumberValidator that will be used in the checkpoint
process to ensure that the sequence number is valid for the shard
being checkpointed.
 2018-10-10 13:02:15 -07:00
akhani18
2609e1ce46 Add a listener to capture task execution in shardConsumer (#417) 
* Add a listener to capture when tasks are executed in the ShardConsumer
 2018-10-10 13:01:41 -07:00
xiaoyu meng
14c68296f0 Introducing HierarchicalShardSyncer inorder to run multiple Schedulers in a JVM (#395) 
* Run multiple instance of scheduler on one JVM

* handling creation of shardSyncer in DynamoDBLeaseManagementFactory and LeaseManagementConfig

* remove multi-threading unit test and do some small refactorings

* refectoring

* deprecate ShardSyncer and use HierarchichalShardSyncer instead; change the order for metricsFactory and HierarchichalShardSyncer in ShardConsumerArgument

* fix typos and use mock object of shardSyncer

* delete improper comments

* fix comments

* remove duplicated comments
 2018-10-09 17:29:59 -07:00
Sahil Palvia
854e316b83 Quick fix for shutdown race issue (#439) 
* Added a synchronized lock in the initialize and shutdown methods
 2018-10-09 13:30:35 -07:00