Default the consumer library to read from latest

Having consumers start at latest record seems like a reasonable default.
This can also be overridden with optional config, make sure that is
documented for users of the library.

Fixes: https://github.com/harlow/kinesis-consumer/issues/86
This commit is contained in:
Harlow Ward 2019-04-09 22:10:07 -07:00
parent 97fe4e66ff
commit f7f98a4bc6
2 changed files with 17 additions and 1 deletions

View file

@ -234,6 +234,22 @@ The [expvar package](https://golang.org/pkg/expvar/) will display consumer count
},
```
### Consumer starting point
Kinesis allows consumers to specify where on the stream they'd like to start consuming from. The default in this library is `LATEST` (Start reading just after the most recent record in the shard).
This can be adjusted by using the `WithShardIteratorType` option in the library:
```go
// override starting place on stream to use TRIM_HORIZON
c, err := consumer.New(
*stream,
consumer.WithShardIteratorType(kinesis.ShardIteratorTypeTrimHorizon)
)
```
[See AWS Docs for more options.](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html)
### Logging
Logging supports the basic built-in logging library or use thrid party external one, so long as

View file

@ -26,7 +26,7 @@ func New(streamName string, opts ...Option) (*Consumer, error) {
// new consumer with no-op checkpoint, counter, and logger
c := &Consumer{
streamName: streamName,
initialShardIteratorType: kinesis.ShardIteratorTypeTrimHorizon,
initialShardIteratorType: kinesis.ShardIteratorTypeLatest,
checkpoint: &noopCheckpoint{},
counter: &noopCounter{},
logger: &noopLogger{