luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
469 commits 48 branches 124 tags 11 MiB
Commit graph

469 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ashwin Giridharan
ff1bee5791 Fix for avoiding premature creation of child lease when other parent lease is not created yet. This can happen in the case of fast reshard operations. 2020-07-14 16:59:31 -07:00
ashwing
5fc55e0c8c
Merge pull request #68 from ychunxue/childShardValidation 
ChildShard validation in Data Fetcher level and error handling in shu…
 2020-07-14 16:33:53 -07:00
Chunxue Yang
85a5423657 Addressing more comments 2020-07-14 15:02:04 -07:00
Chunxue Yang
a0094b0df8 Addressing comments 2020-07-08 12:33:16 -07:00
Renju Radhakrishnan
347ae9eb68
Add very old leases + update lease taker to always emit metrics (#54) 
* Add very old leases + metrics update

* Add conditional emisison for metrics in case of very old leases

* move code blocks around to ensure all metrics are emitted
 2020-07-07 16:58:40 -07:00
Chunxue Yang
b49d8ea8cc ChildShard validation in Data Fetcher level and error handling in shutdownTask 2020-07-07 16:02:12 -07:00
Joshua Kim
74ffd4060c Revert "Fixing bug for multistream not using list shards with filter. (#66)" 
This reverts commit f693311ac8.
 2020-07-01 19:07:50 -04:00
Joshua Kim
f693311ac8
Fixing bug for multistream not using list shards with filter. (#66) 
Fixing bug for mulitstreaming not using list shards with filter.
 2020-07-01 19:01:13 -04:00
ashwing
842c755511
Merge pull request #65 from ashwing/ltr_1_lease_cleanup_logging 
Ltr 1 lease cleanup logging
 2020-06-24 17:15:07 -07:00
Ashwin Giridharan
950faf7475 Fixing condition check 2020-06-24 14:34:01 -07:00
Ashwin Giridharan
b636edd007 ShutdownTask bug fix 2020-06-24 13:07:42 -07:00
Ashwin Giridharan
bd6a7d8b90 Logging to track pending leases to cleanup 2020-06-24 12:43:55 -07:00
Ashwin Giridharan
a2b6019336 Changes to enqueue lease for deletion in potential garbage cleanup scenario. 2020-06-24 12:39:18 -07:00
ashwing
98d723b576
Merge pull request #63 from ashwing/ltr_1_duplicate_recseq_fix 
Fixing Prefetch publisher cache restart issue
 2020-06-23 17:36:16 -07:00
ashwing
1b86184428
Merge pull request #64 from ashwing/ltr_1_leader_stream_sync 
Ltr 1 leader stream sync
 2020-06-23 15:45:04 -07:00
Ashwin Giridharan
60af78f7cb Adding unit test case for validating internal state on initial prefetcher failures 2020-06-23 14:49:15 -07:00
Ashwin Giridharan
67ce7a783d Fixing lease cleanup issues with multistreaming 2020-06-23 03:03:42 -07:00
Ashwin Giridharan
ef39ecd0df Log changes 2020-06-22 23:37:38 -07:00
Ashwin Giridharan
d4f3c0b14a Fixing Prefetch publisher cache restart issue 2020-06-22 23:23:41 -07:00
Ashwin Giridharan
0246e1e852 Only making leader to do the stream sync 2020-06-22 14:30:28 -07:00
ychunxue
db2c22e046
Merge pull request #61 from ychunxue/ltr_1_createChildLeaseFix 
search child shards in DDB instead of in local cache
 2020-06-17 12:10:25 -07:00
ashwing
7bb60b0c69
Merge pull request #62 from ashwing/ltr_1_skip_shardsync_on_empty_stream_shards 
Skipping ddb scan and shard sync for streams that has no shards retur…
 2020-06-17 11:57:48 -07:00
Chunxue Yang
27613763ac search child shards in DDB instead of in local cache 2020-06-17 11:31:22 -07:00
Ashwin Giridharan
8d40e5bdd4 Skipping ddb scan and shard sync for streams that has no shards returned from service 2020-06-17 10:49:57 -07:00
ashwing
50a1fe8922
Merge pull request #60 from ashwing/ltr_1_old_streams_cleanup 
Removing the old streams from active streams list without cleaning up the leases
 2020-06-17 10:40:49 -07:00
Ashwin Giridharan
69a05b409b Removing the old streams from active streams list without cleaning up the leases 2020-06-17 10:25:11 -07:00
ashwing
6021261b55
Merge pull request #59 from Renjuju/cherry-pick-rnf-exception 
[Cherry-pick from multi_stream_2] Empty lease response for ResourceNotFound exceptions (#27)
 2020-06-16 15:44:36 -07:00
Renju Radhakrishnan
4a89c26f6f Empty lease response for ResourceNotFound exceptions (#27) 2020-06-16 13:42:20 -07:00
ashwing
562face833
Merge pull request #55 from ashwing/ltr_1_periodic_auditor_metrics_configs 
Ltr 1 periodic auditor metrics configs
 2020-06-16 10:21:23 -07:00
ashwing
a24e7dcf03
Merge pull request #57 from Renjuju/cherry-pick-leases-list-to-set 
[Cherry-pick] Convert expiredLeases to set (#56)
 2020-06-16 10:19:45 -07:00
Ashwin Giridharan
e7e8196f29 Fixed typo 2020-06-16 10:09:13 -07:00
ashwing
65388573f9
Merge pull request #58 from ashwing/ltr_1_lease_cleanup_fix 
Lease cleanup handling for garbage shard
 2020-06-16 09:10:46 -07:00
Ashwin Giridharan
374e47b208 updating wereChildShardsPresent hint in all cases 2020-06-15 23:13:47 -07:00
Ashwin Giridharan
4d65f56038 Added comments on the garbage cleanup logic - 2 2020-06-15 22:02:33 -07:00
Ashwin Giridharan
7b0dc0d3e5 Added comments on the garbage cleanup logic 2020-06-15 22:00:56 -07:00
Ashwin Giridharan
8ec14baca1 Handlign garbage shard case while delivering lease cleanup 2020-06-15 21:53:30 -07:00
Ashwin Giridharan
b60dd60f35 Code refactoring - 1 2020-06-15 21:20:33 -07:00
Ashwin Giridharan
37281e9493 Deleting lease immediately in RNF Exception 2020-06-15 20:57:55 -07:00
Renju Radhakrishnan
aaf37be0cd Convert expiredLeases to set (#56) 2020-06-15 11:01:44 -07:00
Ashwin Giridharan
c1cbb6cf6d Removing one of the lease clean up strategy as it is no longer required with distributed lease cleanup 2020-06-12 22:22:18 -07:00
Ashwin Giridharan
ce38178399 Caching consumerArn for StreamIdentifier in FanOutRetrievalFactory 2020-06-12 22:20:38 -07:00
Ashwin Giridharan
2530481dba Periodic Shard Sync Auditor - Metrics and Customer auditor configs 2020-06-12 18:12:02 -07:00
ashwing
1aeb8ed9a1
Merge pull request #53 from ashwing/ltr_ddb_scan_opt 
Limiting max page for ddb scan in isEmptyLeaseTable check to be 1
 2020-06-11 18:42:48 -07:00
Ashwin Giridharan
6fc654802c Limiting max page for ddb scan in isEmptyLeaseTable check to be 1 2020-06-11 18:36:42 -07:00
Joshua Kim
c8422745d9
Removing lease cleanup from HierarchicalShardSyncer (#30) 
* Removing lease cleanup from HierarchicalShardSyncer
 2020-06-11 19:56:04 -04:00
Joshua Kim
2e2211a9b7
Merge pull request #31 from ashwing/ShutdownTaskCleanup 
* Adding LeaseCleanupManager
* Added lease cleanup unit tests
* Adding lease cleanup path to shard consumer shutdown
 2020-06-11 19:48:58 -04:00
Joshua Kim
445dd106b2 Adding logging 2020-06-11 19:48:24 -04:00
Joshua Kim
dc078ead39 Reverting pom change 2020-06-10 19:49:06 -04:00
Joshua Kim
2f1838483c Adding Lease cleanup in shutdown task. 2020-06-10 19:45:11 -04:00
Joshua Kim
c7cdbd5d8b pom change 2020-06-10 18:57:53 -04:00