Having an additional Client has added some confusion (https://github.com/harlow/kinesis-consumer/issues/45) on how to provide a
custom kinesis client. Allowing `WithClient` to accept a Kinesis client
it cleans up the interface.
Major changes:
* Remove the Client wrapper; prefer using kinesis client directly
* Change `ScanError` to `ScanStatus` as the return value isn't necessarily an error
Note: these are breaking changes, if you need last stable release please see here: https://github.com/harlow/kinesis-consumer/releases/tag/v0.2.0
A closed shard returns the last sequence number and no error. The current implementation leads to an infinite loop if the shard is closed. NextShardIterator checking is enough. That's why I remove the getShardIterator call
- scanError.Error is removed since it is not used.
- session.New() is deprecated, The NewKinesisClient's function signature change so it can
returns the error from the NewSession().
* remove ValidateCheckpoint
* update for checkpoint can not customize retryer
* implement the scan error as in PR 44
* at least log if record processor has error
* mistakenly removed this line
* propage error up. ignore invalid state
Add interval flush for DDB checkpoint
* Allow checkpointing on a specified interval
* Add shutdown method to checkpoint to force flush
Minor changes:
* Swap order of input params for checkpoint (app, table)
Addresses: https://github.com/harlow/kinesis-consumer/issues/39
Testing the components of the consumer where proving difficult because
the consumer code was so tightly coupled with the Kinesis client
library.
* Extract the concept of Client interface
* Create default client w/ kinesis connection
* Test with fake client to avoid round trip to kinesis
The Checkpoint functionality is an important part of the library and
previously it wasn't obvious that the Consumer was defaulting to Redis
for this functionality.
* Add Checkpoint as required param for new consumer
Major changes:
* Remove intermediate batching of kinesis records
* Call the callback func with each record
* Use functional options for config
https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
Minor changes:
* update README messaging about Kinesis -> Firehose functionality
* remove unused buffer and emitter code
