luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
483 commits 48 branches 124 tags 11 MiB
Commit graph

483 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ashwin Giridharan
bdf019748f Add conditional check while updating the lease table meta info 2020-07-27 16:26:51 -07:00
ashwing
e235777c17
Merge pull request #83 from ashwing/ltr_1_shardsync_executor_overload_fix 
Avoiding ShardSync Task sleep when we skip the shard sync due to no s…
 2020-07-22 12:16:47 -07:00
Ashwin Giridharan
75fe6060f6 Adding logging around the shard sync executor queuesize 2020-07-22 11:37:25 -07:00
Ashwin Giridharan
8aec062e64 Avoiding ShardSync Task sleep when we skip the shard sync due to no shards found 2020-07-22 10:36:22 -07:00
Joshua Kim
ff703459e1
Exposing interface for getting child shards (#81) 
* Exposing interface for getting child shards

* javadocs

* more java docs

* Addressing comments

* Tuning random max range

Co-authored-by: Joshua Kim <kimjos@amazon.com>
 2020-07-17 18:32:50 -04:00
Renju Radhakrishnan
c68ab705bd
Fix NonGreedyTake integration test and add extra integration test (#80) 
* Fix NonGreedyTake integration test and add extra integration test for very old leases

* Fix spacing in comment
 2020-07-17 14:48:25 -07:00
ashwing
afd0f7c08a
Merge pull request #77 from ashwing/ltr_1_shutdown_probability_based_exception 
Breaking loop on ShutdownException using 10-faced dice
 2020-07-16 20:42:26 -07:00
ashwing
f46bd444d4
Merge pull request #78 from ashwing/ltr_1_rnf_log 
Log details of ResourceNotFoundException while cleaning up leases
 2020-07-16 20:41:39 -07:00
Ashwin Giridharan
4c899b2930 Log details of ResourceNotFoundException while cleaning up leases 2020-07-16 20:34:48 -07:00
Ashwin Giridharan
60b6066189 Breaking loop on ShutdownException using 10-faced dice 2020-07-16 12:14:53 -07:00
Ashwin Giridharan
7b704246e2 Fixing NPE in log 2020-07-15 14:52:12 -07:00
ashwing
4457ebc350
Merge pull request #74 from ashwing/ltr_1_fix_for_premature_childshard_creation 
Ltr 1 fix for premature childshard creation
 2020-07-15 11:35:07 -07:00
Ashwin Giridharan
264e5bd4aa Shutdown task to throw retryable exception intermittently to allow parent shard lease detection in merge cases. 2020-07-15 10:08:50 -07:00
Ashwin Giridharan
d9f5557ff5 Added unit test cases 2020-07-14 16:59:31 -07:00
Ashwin Giridharan
ff1bee5791 Fix for avoiding premature creation of child lease when other parent lease is not created yet. This can happen in the case of fast reshard operations. 2020-07-14 16:59:31 -07:00
ashwing
5fc55e0c8c
Merge pull request #68 from ychunxue/childShardValidation 
ChildShard validation in Data Fetcher level and error handling in shu…
 2020-07-14 16:33:53 -07:00
Chunxue Yang
85a5423657 Addressing more comments 2020-07-14 15:02:04 -07:00
Chunxue Yang
a0094b0df8 Addressing comments 2020-07-08 12:33:16 -07:00
Renju Radhakrishnan
347ae9eb68
Add very old leases + update lease taker to always emit metrics (#54) 
* Add very old leases + metrics update

* Add conditional emisison for metrics in case of very old leases

* move code blocks around to ensure all metrics are emitted
 2020-07-07 16:58:40 -07:00
Chunxue Yang
b49d8ea8cc ChildShard validation in Data Fetcher level and error handling in shutdownTask 2020-07-07 16:02:12 -07:00
Joshua Kim
74ffd4060c Revert "Fixing bug for multistream not using list shards with filter. (#66)" 
This reverts commit f693311ac8.
 2020-07-01 19:07:50 -04:00
Joshua Kim
f693311ac8
Fixing bug for multistream not using list shards with filter. (#66) 
Fixing bug for mulitstreaming not using list shards with filter.
 2020-07-01 19:01:13 -04:00
ashwing
842c755511
Merge pull request #65 from ashwing/ltr_1_lease_cleanup_logging 
Ltr 1 lease cleanup logging
 2020-06-24 17:15:07 -07:00
Ashwin Giridharan
950faf7475 Fixing condition check 2020-06-24 14:34:01 -07:00
Ashwin Giridharan
b636edd007 ShutdownTask bug fix 2020-06-24 13:07:42 -07:00
Ashwin Giridharan
bd6a7d8b90 Logging to track pending leases to cleanup 2020-06-24 12:43:55 -07:00
Ashwin Giridharan
a2b6019336 Changes to enqueue lease for deletion in potential garbage cleanup scenario. 2020-06-24 12:39:18 -07:00
ashwing
98d723b576
Merge pull request #63 from ashwing/ltr_1_duplicate_recseq_fix 
Fixing Prefetch publisher cache restart issue
 2020-06-23 17:36:16 -07:00
ashwing
1b86184428
Merge pull request #64 from ashwing/ltr_1_leader_stream_sync 
Ltr 1 leader stream sync
 2020-06-23 15:45:04 -07:00
Ashwin Giridharan
60af78f7cb Adding unit test case for validating internal state on initial prefetcher failures 2020-06-23 14:49:15 -07:00
Ashwin Giridharan
67ce7a783d Fixing lease cleanup issues with multistreaming 2020-06-23 03:03:42 -07:00
Ashwin Giridharan
ef39ecd0df Log changes 2020-06-22 23:37:38 -07:00
Ashwin Giridharan
d4f3c0b14a Fixing Prefetch publisher cache restart issue 2020-06-22 23:23:41 -07:00
Ashwin Giridharan
0246e1e852 Only making leader to do the stream sync 2020-06-22 14:30:28 -07:00
ychunxue
db2c22e046
Merge pull request #61 from ychunxue/ltr_1_createChildLeaseFix 
search child shards in DDB instead of in local cache
 2020-06-17 12:10:25 -07:00
ashwing
7bb60b0c69
Merge pull request #62 from ashwing/ltr_1_skip_shardsync_on_empty_stream_shards 
Skipping ddb scan and shard sync for streams that has no shards retur…
 2020-06-17 11:57:48 -07:00
Chunxue Yang
27613763ac search child shards in DDB instead of in local cache 2020-06-17 11:31:22 -07:00
Ashwin Giridharan
8d40e5bdd4 Skipping ddb scan and shard sync for streams that has no shards returned from service 2020-06-17 10:49:57 -07:00
ashwing
50a1fe8922
Merge pull request #60 from ashwing/ltr_1_old_streams_cleanup 
Removing the old streams from active streams list without cleaning up the leases
 2020-06-17 10:40:49 -07:00
Ashwin Giridharan
69a05b409b Removing the old streams from active streams list without cleaning up the leases 2020-06-17 10:25:11 -07:00
ashwing
6021261b55
Merge pull request #59 from Renjuju/cherry-pick-rnf-exception 
[Cherry-pick from multi_stream_2] Empty lease response for ResourceNotFound exceptions (#27)
 2020-06-16 15:44:36 -07:00
Renju Radhakrishnan
4a89c26f6f Empty lease response for ResourceNotFound exceptions (#27) 2020-06-16 13:42:20 -07:00
ashwing
562face833
Merge pull request #55 from ashwing/ltr_1_periodic_auditor_metrics_configs 
Ltr 1 periodic auditor metrics configs
 2020-06-16 10:21:23 -07:00
ashwing
a24e7dcf03
Merge pull request #57 from Renjuju/cherry-pick-leases-list-to-set 
[Cherry-pick] Convert expiredLeases to set (#56)
 2020-06-16 10:19:45 -07:00
Ashwin Giridharan
e7e8196f29 Fixed typo 2020-06-16 10:09:13 -07:00
ashwing
65388573f9
Merge pull request #58 from ashwing/ltr_1_lease_cleanup_fix 
Lease cleanup handling for garbage shard
 2020-06-16 09:10:46 -07:00
Ashwin Giridharan
374e47b208 updating wereChildShardsPresent hint in all cases 2020-06-15 23:13:47 -07:00
Ashwin Giridharan
4d65f56038 Added comments on the garbage cleanup logic - 2 2020-06-15 22:02:33 -07:00
Ashwin Giridharan
7b0dc0d3e5 Added comments on the garbage cleanup logic 2020-06-15 22:00:56 -07:00
Ashwin Giridharan
8ec14baca1 Handlign garbage shard case while delivering lease cleanup 2020-06-15 21:53:30 -07:00