luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial FAQ, with hopefully more contributions in the future. (#1202)

Browse source
This commit is contained in:
stair 2023-08-30 13:07:23 -04:00 committed by GitHub
parent 78b565fa9b
commit aabcbaf4b7
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 29 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/FAQ.md Normal file
View file

@ -0,0 +1,29 @@
# Frequently Asked Questions (FAQ)
## Stream Modality
Questions related to stream modality (e.g., [MultiStreamTracker][multi-stream-tracker]).
### What is the impact of transitioning my KCL app from single-stream to multi-stream?
This answer assumes the [StreamTracker][stream-tracker] implementation is being changed.
From KCL's perspective, there is no modality change by decreasing a multi-stream tracker from `N` streams to `1` stream.
The DDB `leaseKey`, used to persist metadata including lease checkpoint, has a modality-dependent format:
| Modality | `leaseKey` Format |
| --- | --- |
| single-stream | `<shardId>` |
| multi-stream | `<accountId>:<streamName>:<streamCreationTimestamp>:<shardId>` |
Transitioning an app -- either from single- to multi-, or vice versa -- creates a backwards-incompatible expectation on the `leaseKey`.
As a result, a KCL app will be blind to any `leaseKey`, and its checkpoint, that does not match the expected format.
## Resources
For additional information, please consider reading:
* https://docs.aws.amazon.com/streams/latest/dev/kcl-migration.html
* https://docs.aws.amazon.com/streams/latest/dev/shared-throughput-kcl-consumers.html
[multi-stream-tracker]: /amazon-kinesis-client/src/main/java/software/amazon/kinesis/processor/MultiStreamTracker.java
[stream-tracker]: /amazon-kinesis-client/src/main/java/software/amazon/kinesis/processor/StreamTracker.java