Ashwin Giridharan
c1cbb6cf6d
Removing one of the lease clean up strategy as it is no longer required with distributed lease cleanup
2020-06-12 22:22:18 -07:00
Ashwin Giridharan
ce38178399
Caching consumerArn for StreamIdentifier in FanOutRetrievalFactory
2020-06-12 22:20:38 -07:00
Ashwin Giridharan
2530481dba
Periodic Shard Sync Auditor - Metrics and Customer auditor configs
2020-06-12 18:12:02 -07:00
ashwing
1aeb8ed9a1
Merge pull request #53 from ashwing/ltr_ddb_scan_opt
...
Limiting max page for ddb scan in isEmptyLeaseTable check to be 1
2020-06-11 18:42:48 -07:00
Ashwin Giridharan
6fc654802c
Limiting max page for ddb scan in isEmptyLeaseTable check to be 1
2020-06-11 18:36:42 -07:00
Joshua Kim
c8422745d9
Removing lease cleanup from HierarchicalShardSyncer ( #30 )
...
* Removing lease cleanup from HierarchicalShardSyncer
2020-06-11 19:56:04 -04:00
Joshua Kim
2e2211a9b7
Merge pull request #31 from ashwing/ShutdownTaskCleanup
...
* Adding LeaseCleanupManager
* Added lease cleanup unit tests
* Adding lease cleanup path to shard consumer shutdown
2020-06-11 19:48:58 -04:00
Joshua Kim
445dd106b2
Adding logging
2020-06-11 19:48:24 -04:00
Joshua Kim
dc078ead39
Reverting pom change
2020-06-10 19:49:06 -04:00
Joshua Kim
2f1838483c
Adding Lease cleanup in shutdown task.
2020-06-10 19:45:11 -04:00
Joshua Kim
c7cdbd5d8b
pom change
2020-06-10 18:57:53 -04:00
ashwing
61aeae3ea1
Merge pull request #48 from ashwing/ltr_1_hsyncperstreamconfig
...
Hierachical stream syncer per stream config changes
2020-06-10 10:35:24 -07:00
ashwing
291d1b4059
Merge pull request #44 from ashwing/ltr_1_ms_consumerarn
...
Dedicated consumerArn for streams in multiStream mode
2020-06-09 18:18:21 -07:00
Ashwin Giridharan
b323e7c487
Introducing dedicated shard syncer for each of the streamconfig
2020-06-09 17:14:02 -07:00
Ashwin Giridharan
700c7a563a
Adding unit test case
2020-06-07 21:17:24 -07:00
Ashwin Giridharan
b5d0301b31
Adding stream information to Fanout Consumer Registration logs
2020-06-03 01:08:13 -07:00
Ashwin Giridharan
f69398a2b2
Adding dedicated consumerArn support for streams in multistreaming mode
2020-06-03 00:48:11 -07:00
Micah Jaffe
113029e33c
Update lease sync algorithm to create leases for no more than one level of the shard hierarchy ( #41 )
...
* Update NonEmptyLeaseTableSynchronizer algorithm to create leases for no more than one level of the shard hierarchy
* Add and fix unit tests for LATEST, TRIM_HORIZON, AT_TIMESTAMP
* Update unit tests
* Fix edge case of reading from TRIM with partial lease for one child shard
* Revert child shards optimization for TRIM/TIMESTAMP case
2020-05-31 10:28:05 -07:00
Ashwin Giridharan
f2ba3bcd2f
Changes for partial lease table hole detection, missing hash range detection and recovery
2020-05-29 09:10:14 -07:00
Ashwin Giridharan
67d2b082fd
Fixing retry logic
2020-05-26 11:10:10 -07:00
Ashwin Giridharan
08ca1b61bc
Changing test case name
2020-05-25 13:37:10 -07:00
Ashwin Giridharan
72a6d5e084
Adding more unit test cases and fixing an edge case
2020-05-25 13:25:43 -07:00
Ashwin Giridharan
fc4781e347
Addressed review comments
2020-05-21 00:57:24 -07:00
ychunxue
52f1be5c67
Merge pull request #32 from ychunxue/ltr_1_updateLeaseFix
...
Update Parent lease fix
2020-05-07 14:58:24 -07:00
Ashwin Giridharan
02ea8cd70f
Added TODO for childshards update fix
2020-05-07 01:36:51 -07:00
Ashwin Giridharan
9115f2000b
Logic to auto fill the missing hashranges and lease recovery. Added more unit test cases
2020-05-07 01:28:56 -07:00
Chunxue Yang
80cdf0df6f
Update Parent lease fix
2020-05-06 21:12:50 -07:00
Ashwin Giridharan
9e97edd273
Reusing code while determining leaskekey
2020-05-06 11:06:23 -07:00
Ashwin Giridharan
a6922d9d7e
Hash range hole confidence check
2020-05-06 10:45:31 -07:00
Ashwin Giridharan
eb00229602
Hole detection logic
2020-05-06 10:45:31 -07:00
Ashwin Giridharan
5cd40e4718
Lease Recovery and Blockoing parent bug fix
2020-05-06 10:45:31 -07:00
Ashwin Giridharan
eb7a60ec62
Fixing a metrics bug
2020-05-06 10:42:34 -07:00
Ashwin Giridharan
d28d26c635
Adding java docs
2020-05-06 10:41:12 -07:00
Ashwin Giridharan
e86b1d1f01
Adding shouldCleanupLeasesForDeletedStreams option to FormerStreamsLeasesDeletionStrategy and introducing OnlyDeletedStreamsLeasesCleanupStrategy.
2020-05-05 18:44:06 -07:00
ychunxue
8e4b8d789b
Merge pull request #26 from ychunxue/ltr_1_lease_parameter
...
leaseKey parameter fix and logging change for shutdownTask
2020-05-05 17:36:24 -07:00
Chunxue Yang
760c52d7bc
getLeaseKey
2020-05-05 17:31:08 -07:00
Chunxue Yang
1036006bb4
leaseKey parameter fix and logging change for shutdownTask
2020-05-05 15:44:45 -07:00
Ashwin Giridharan
c479984fb4
Hashrange persistence changes
2020-05-05 12:58:16 -07:00
Ashwin Giridharan
a73701ff87
Lease I/F changes and drop lease fix
2020-04-30 17:37:13 -07:00
Ashwin Giridharan
4ce0591e79
Adding hashkeyRange back
2020-04-29 15:59:53 -07:00
Ashwin Giridharan
4a323b8f1c
Fixing a metrics bug
2020-04-29 15:56:15 -07:00
Ashwin Giridharan
bdec3bd66d
Fixing comparison bug and addressing review comments
2020-04-29 15:23:14 -07:00
Renju Radhakrishnan
f69e9cf3ba
Add getResponse override for ShardDetector ( #19 ) ( #21 )
2020-04-28 16:32:29 -07:00
Ashwin Giridharan
07c9529c14
doc comments
2020-04-28 15:58:50 -07:00
Ashwin Giridharan
4d058ecf51
Persisting hashrange in lease table
2020-04-28 15:53:22 -07:00
Ashwin Giridharan
038524e0b1
Introducing lease deletion strategy for multistreaming
2020-04-24 17:02:55 -07:00
Ashwin Giridharan
167ecfb08c
Add licensing information
2020-04-24 17:02:55 -07:00
Ashwin Giridharan
61e500d4fd
Adding logging for streanms to be deleted
2020-04-24 17:02:55 -07:00
Ashwin Giridharan
596e3ee797
Exposing interface method to set the wait time for old stream deletion. Adding license details to new classes
2020-04-24 17:02:55 -07:00
Ashwin Giridharan
45387bfd74
Adding logic to garbage collect active streams from stale streams list.
2020-04-24 17:02:54 -07:00
