Commit graph

8 commits

Author SHA1 Message Date
Harlow Ward
2f58b136fe Add dummy users data for producing onto stream
Pulled from: https://github.com/awslabs/amazon-kinesis-connectors
2018-12-30 07:28:18 -08:00
Harlow Ward
fb98fbe244
Remove the client wrapper (#58)
Having an additional Client has added some confusion (https://github.com/harlow/kinesis-consumer/issues/45) on how to provide a
custom kinesis client. Allowing `WithClient` to accept a Kinesis client
it cleans up the interface.

Major changes:

* Remove the Client wrapper; prefer using kinesis client directly
* Change `ScanError` to `ScanStatus` as the return value isn't necessarily an error

Note: these are breaking changes, if you need last stable release please see here: https://github.com/harlow/kinesis-consumer/releases/tag/v0.2.0
2018-07-28 22:53:33 -07:00
Prometheus
992cc42419 DDB uses default AWS config settings to ping table; won't work with WithDyanmoClient. Misc update on example and README (#50) 2018-06-01 16:14:42 -07:00
Harlow Ward
89570130f5 Leverage bigger batchsize when seeding example data 2017-11-26 15:59:17 -08:00
Harlow Ward
4d6a85e901 Make the Checkpoint a required input for Consumer
The Checkpoint functionality is an important part of the library and
previously it wasn't obvious that the Consumer was defaulting to Redis
for this functionality.

* Add Checkpoint as required param for new consumer
2017-11-21 08:58:16 -08:00
Harlow Ward
130c78456c
Simplify the consumer experience (#35)
Major changes:

* Remove intermediate batching of kinesis records
* Call the callback func with each record
* Use functional options for config 

https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis

Minor changes:

* update README messaging about Kinesis -> Firehose functionality
* remove unused buffer and emitter code
2017-11-20 08:21:40 -08:00
Harlow Ward
ceca88b96a Add required fields to Config
Taking some inspiration from:
https://github.com/tj/go-kinesis/blob/master/kinesis.go#L50-L75
2016-05-07 18:10:31 -07:00
Harlow Ward
f4c8d8090d Use TJ's Kinesis producer library 2016-05-01 16:54:47 -07:00