luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update kcl_3x_deep-dive.md

Browse source
This commit is contained in:
Minu Hong 2025-04-11 16:11:18 -07:00 committed by GitHub
parent 5f0efd2333
commit b6317bf948
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/kcl_3x_deep-dive.md
View file

@ -1,3 +1,13 @@
## Introduction
This document
- [Overview: Lease reassignments in KCL 3.x](#overview-lease-reassignments-in-kcl-3x)
- [Motivation for moving from distributed to leader-based protocol](#motivation-for-moving-from-distributed-to-leader-based-protocol)
- [How KCL 3.x manages leader election]
- [How KCL 3.x captures throughput per shard]
- [How KCL 3.x captures CPU utilization of worker hosts]
- [How KCL 3.x performs lease assignments]
## Overview: Lease reassignments in KCL 3.x
KCL 3.x rebalances leases based on the load (CPU utilization) on each worker with a goal of even distribution of load across workers. Unlike KCL 2.x where all workers scan the entire lease table to identify the lease assignment status and pick up unassigned leases or steal leases from existing workers, KCL 3.x achieves it by electing a leader worker scanning the entire lease table and do the lease assignment for the entire KCL workers. The following is the key differences in KCL 3.x.