Harlow Ward
7fccc9da9f
Update function name to represent functionality
2016-04-30 09:14:25 -07:00
Harlow Ward
c29698550f
Add config options to Consumer
...
The Firehose service can take a max batch size of 500. While created the
example the need for finer grained configuration was necessary.
2016-02-09 22:31:15 -08:00
Harlow Ward
aae268108e
Move emitters to separate packages
2016-02-08 19:39:09 -08:00
Harlow Ward
caa2fc0b0a
Add Kinesis Firehose example
2016-02-08 13:21:54 -08:00
Harlow Ward
f0e6461cb6
Refactor to use handler func
...
The previous pipeline model required a lot of setup and abstracted away
the processing of records. By passing a HandlerFunc to the consumer we
keep the business logic of processing of records closer to the use of
the consumer.
* Add refactoring note and SHA to README
2016-02-06 17:50:17 -08:00
Harlow Ward
509f68de89
Add error checking for tmp file
2016-01-09 09:52:47 -08:00
Harlow Ward
d542fa996f
Use AWS SDK and concurrency
2016-01-09 09:46:28 -08:00
Harlow Ward
0d5e9b7b02
Use AWS SDK for S3 Emitter
...
* Handle retries for S3 Emitter
* Update Pipeline to use AWS backoff
* Leverage aws error kit in recoverable error helpers
2015-08-16 17:52:10 -07:00
Harlow Ward
18173842fb
Use AWS SDK
...
Limit the amount of library dependencies by using the official AWS SDK.
https://github.com/harlow/kinesis-connectors/issues/19
2015-08-15 23:17:49 -07:00
Harlow Ward
3d9e6e2485
Add Redshift Pipeline example
2015-05-25 23:18:14 -07:00
Harlow Ward
cd54569c61
Use go-kit Log interface
2015-05-25 22:40:40 -07:00
Harlow Ward
25e390e8b4
Add sample pipeline
...
* Reduce the size of main README in favor of adding functioning examples
* Add S3 Pipeline exmaple
* Add example of seeing the stream
2015-05-23 15:27:40 -07:00
