bencvdb
f2dd16257d
fix(worker): actually throw the error
2021-07-06 16:47:21 -07:00
bencvdb
a8551d5389
refactor(worker): consolidate exception checking logic
2021-07-06 16:40:46 -07:00
bencvdb
981bacd512
fix(worker): fail on incomplete garbage lease
2021-07-06 10:25:56 -07:00
bencvdb
c8a1797b7e
fix(worker): fail on TrimmedDataAccessException (101797)
2021-07-06 10:25:56 -07:00
zekailiu
6b287424d9
fix(kinesis_proxy): fix AmazonKinesis client class name check
2019-12-23 17:28:23 -08:00
George Fraser
124a602515
Merge pull request #3 from fivetran/shade
...
Move to com.fivetran.external
2019-12-06 07:27:26 -08:00
George Fraser
4aefa9ba1a
Stray reference to old package name
2019-12-05 21:17:57 -08:00
George Fraser
f987cc7f9a
Move to com.fivetran.external
2019-12-05 21:07:27 -08:00
bencvdb
88b6ff2f30
Merge pull request #2 from fivetran/throw_insufficent_permissions_error
...
feature(worker): halt worker on permissions error
2019-07-01 14:47:06 -07:00
bencvdb
a807444f57
fix(Worker): align finally
2019-07-01 14:46:35 -07:00
bencvdb
ce875f3d48
feature(worker): remove unnecessary functions and permissions
2019-06-27 09:07:17 -07:00
bencvdb
d2f95cb5ed
feature(worker): catch errors from runProcessLoop and shutdown
2019-06-26 13:51:50 -07:00
bencvdb
9e102af3dd
feature(worker): make Worker#getLeaseCoordinator public
2019-06-26 09:35:59 -07:00
bencvdb
08bf7f7000
feature(worker): remove RuntimeException wrapping
2019-06-25 15:01:54 -07:00
bencvdb
b8861e50b1
feature(worker): throw and handle AmazonDynamDBException
2019-06-25 14:28:02 -07:00
bencvdb
06652d8f88
feature(worker): throw insufficent permissions error
2019-06-25 11:03:14 -07:00
glarwood
4ffc50a8ff
refactor(Worker): update error message
2019-03-22 18:10:29 +00:00
glarwood
8642f0b40b
feat(StreamRecordProcessingError): add new error to stop worker
2019-03-22 17:58:14 +00:00
glarwood
f0230c9040
refactor(Worker): allow worker to fail after error in processing records
2019-03-22 15:17:31 +00:00
glarwood
913f2e9377
refactor(Worker): remove exits after initial lease completion code & default time limit
2019-03-20 20:07:44 +00:00
glarwood
0a4fc8bef0
refactor(ProcessTask): expose shardId
2019-03-20 18:30:39 +00:00
glarwood
0f78ff0bae
fix(Worker): only save lease keys
2019-03-20 16:57:19 +00:00
glarwood
327f0722f5
refactor(Worker): exit after initial leases complete
2019-03-20 15:59:31 +00:00
Gabriel Larwood
08c45f0e42
Merge pull request #1 from fivetran/exit-when-shard-complete
...
refactor(Worker): shutdown when shardInfoShardConsumerMap is empty
2019-03-05 21:06:54 -08:00
glarwood
1a617249f4
refactor(Worker): make Fivetran specific code configurable
2019-03-05 21:04:52 -08:00
glarwood
14fb8e2703
refactor(Worker): shutdown when shardInfoShardConsumerMap is empty
2019-03-05 20:34:58 -08:00
glarwood
6fd2276556
refactor(AWSCredentialsProviderPropertyValueDecoder): cast nameAndArgs as Object[]
2018-12-13 15:04:34 -08:00
glarwood
f42037eefd
fix(GracefulShutdownContext): remove non-null requirement
2018-12-13 12:12:55 -08:00
glarwood
c1b1f9b74d
refactor(src): delombok the src
2018-12-13 11:57:02 -08:00
glarwood
3a91015bba
refactor(Worker): throws exception after MAX_RETRIES on runProcessLoop
2018-12-12 14:27:38 -08:00
glarwood
43bbeb5fe6
revert to release 1.9.3
2018-12-12 13:58:35 -08:00
glarwood
7417524649
refactor(Worker): add retry logic on runProcessLoop. Throws exception after MAX_RETRIES
2018-12-11 19:46:40 -08:00
glarwood
a116817710
Revert back to 0a91e6faa5
2018-12-11 18:53:20 -08:00
Justin Pfifer
a05e22f782
Release 2.0.5 of the Amazon Kinesis Client for Java ( #465 )
2018-11-12 08:54:04 -08:00
Justin Pfifer
f52f2559ed
Remove a possible deadlock on polling queue fill ( #462 )
...
* Remove a possible deadlock on polling queue fill
Adding new items to the receive queue for the PrefetchRecordsPublisher
when at capacity would deadlock retrievals as it was already holding
a lock on this.
The method addArrivedRecordsInput did not need to be synchronized on
this as it didn't change any of the protected
state (requestedResponses). There is a call to drainQueueForRequests
immediately after the addArrivedRecordsInput that will ensure newly
arrived data is dispatched.
This fixes #448
* Small fix on the reasoning comment
* Adjust the test to act more like the ShardConsumer
The ShardConsuemr, which is the principal user of the
PrefetchRecordsPublisher, uses RxJava to consume from publisher. This
test uses RxJava to consume, and notifies the test thread once
MAX_ITEMS * 3 have been received. This ensures that we cycle through
the queue at least 3 times.
* Removed the upper limit on the retrievals
The way RxJava's request management makes it possible that more
requests than we might expect can happen.
2018-11-07 16:33:49 -08:00
Sahil Palvia
b83a32b492
Making configurations consistent in entire package ( #453 )
2018-10-25 10:40:35 -07:00
Sahil Palvia
9f9620354e
Adding travis and codebuild badges to readme for v2.x branch ( #456 )
2018-10-25 08:00:16 -07:00
Sahil Palvia
6bd63f70dd
Updating BuildStatus badge ( #452 )
...
* Adding new BuildStatus badges.
* Updating to show single build status
2018-10-23 12:32:31 -07:00
Sahil Palvia
df6d25a201
Updating version to 2.0.5-SNAPSHOT ( #450 )
2018-10-23 08:09:59 -07:00
Sahil Palvia
eb7b3fd1bb
Release 2.0.4 of the Amazon Kinesis Client for Java ( #447 )
...
Release 2.0.4 of the Amazon Kinesis Client for Java
2018-10-18 10:32:23 -07:00
Justin Pfifer
f2fb9ead0d
Added a sequence number validator to ensure safer checkpoints ( #432 )
...
Added the SequenceNumberValidator that will be used in the checkpoint
process to ensure that the sequence number is valid for the shard
being checkpointed.
2018-10-10 13:02:15 -07:00
akhani18
2609e1ce46
Add a listener to capture task execution in shardConsumer ( #417 )
...
* Add a listener to capture when tasks are executed in the ShardConsumer
2018-10-10 13:01:41 -07:00
xiaoyu meng
14c68296f0
Introducing HierarchicalShardSyncer inorder to run multiple Schedulers in a JVM ( #395 )
...
* Run multiple instance of scheduler on one JVM
* handling creation of shardSyncer in DynamoDBLeaseManagementFactory and LeaseManagementConfig
* remove multi-threading unit test and do some small refactorings
* refectoring
* deprecate ShardSyncer and use HierarchichalShardSyncer instead; change the order for metricsFactory and HierarchichalShardSyncer in ShardConsumerArgument
* fix typos and use mock object of shardSyncer
* delete improper comments
* fix comments
* remove duplicated comments
2018-10-09 17:29:59 -07:00
Sahil Palvia
854e316b83
Quick fix for shutdown race issue ( #439 )
...
* Added a synchronized lock in the initialize and shutdown methods
2018-10-09 13:30:35 -07:00
Sahil Palvia
0326e217f6
Updating version to 2.0.4-SNAPSHOT ( #438 )
2018-10-09 11:55:50 -07:00
shask-amazon
31ab0af901
Added an API on LeaseCoordinator and LeaseTaker to get all leases for… ( #428 )
...
* Added an API on LeaseCoordinator and LeaseTaker to get all leases for the application
2018-10-09 07:56:13 -07:00
Justin Pfifer
e972617bfc
Release note for release 2.0.3 ( #436 )
...
Added release notes, and changelog for the 2.0.3 changes.
Made the warning clearer that it only affects the 2.0 version of the
Amazon Kinesis Client
2018-10-08 15:51:06 -07:00
Justin Pfifer
8e6a8c7da3
Advance version of the AWS SDK to 2.0.6 ( #434 )
...
Fixes a bug when making SubscribeToShard requests over HTTP 1.1.
Using HTTP 1.1 for SubscribeToShard isn't supported, and may be
break at any time.
2018-10-04 10:40:31 -07:00
Sahil Palvia
9e420d83e4
Fixing issue with KinesisClientUtil class ( #433 )
...
* Passing HttpClientBuilder to the client instead of HttpClient
2018-10-03 08:46:34 -07:00
Justin Pfifer
e86bf3d7f3
Revert experimental features from master ( #431 )
...
Reverted 3 commits:
Revert "Change version number to 2.0.3-experimental"
Revert: 54c171dc2a#401 )"
Revert: 592499f7bc#400 )"
Revert: 01f5db8049
2018-10-02 14:50:25 -07:00
