luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

FAQ: Augmented "What is the impact ..." (#1205)

Browse source
This commit is contained in:
stair 2023-09-07 13:46:35 -04:00 committed by GitHub
parent 0aff42f8fe
commit 7899820cb1
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/FAQ.md
View file

@ -1,5 +1,7 @@
# Frequently Asked Questions (FAQ) # Frequently Asked Questions (FAQ)
---
## Stream Modality ## Stream Modality
Questions related to stream modality (e.g., [MultiStreamTracker][multi-stream-tracker]). Questions related to stream modality (e.g., [MultiStreamTracker][multi-stream-tracker]).
@ -18,6 +20,15 @@ The DDB `leaseKey`, used to persist metadata including lease checkpoint, has a m
Transitioning an app -- either from single- to multi-, or vice versa -- creates a backwards-incompatible expectation on the `leaseKey`. Transitioning an app -- either from single- to multi-, or vice versa -- creates a backwards-incompatible expectation on the `leaseKey`.
As a result, a KCL app will be blind to any `leaseKey`, and its checkpoint, that does not match the expected format. As a result, a KCL app will be blind to any `leaseKey`, and its checkpoint, that does not match the expected format.
For leases that don't exist in the expected format, processing may start from the default checkpoint (e.g., `LATEST`).
As an example of potential impact from switching modality, assume `LATEST` is the default initial position in stream.
When a KCL application's modality is switched, stream processing will start reading at this initial position ignoring the checkpoints from the previous modality.
The impact of this is that any records written to the stream between restarting the KCL app will not be processed.
If `TRIM_HORIZON` is used instead upon restarting the application, the application will start reading from its initial position solving for the gap in consumption of data but potentially resulting in duplicate consumption of records.
Thus, please make sure that your application can handle this.
---
## Resources ## Resources