Commit graph

12 commits

Author SHA1 Message Date
Harlow Ward
911282363e Update go dep and prune unused packages 2018-07-28 22:58:40 -07:00
Vincent
a1239221d8 Ignore IDE files and fix code based Gometalinter (#63)
- scanError.Error is removed since it is not used.
- session.New() is deprecated, The NewKinesisClient's function signature change so it can
  returns the error from the NewSession().
2018-07-24 20:10:38 -07:00
Emanuel Ramos
b7be26418a Add postgres checkpoint implementation (#55) 2018-06-17 19:27:10 -07:00
Harlow Ward
130c78456c
Simplify the consumer experience (#35)
Major changes:

* Remove intermediate batching of kinesis records
* Call the callback func with each record
* Use functional options for config 

https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis

Minor changes:

* update README messaging about Kinesis -> Firehose functionality
* remove unused buffer and emitter code
2017-11-20 08:21:40 -08:00
Harlow Ward
fedb6812fb Add checkpoint interface for custom checkpoints (#29)
To allow other checkpoint backends we extract a checkpoint interface
which future checkpoints can implement.

* Add checkpoint interface for custom checkpoints
* Create RedisCheckpoint to implement checkpoint interface
* Swap out hosie redis library for go-redis

Minor changes

* Allow configuration of Redis endpoint with env var `REDIS_URL`
* Replace gvt with govendor
2016-12-04 00:08:06 -08:00
Harlow Ward
c7a69e2c92 Add benchmark test to buffer 2016-05-01 10:42:51 -07:00
Rob O'Dwyer
179e9502b0 add gvt manifest 2016-02-09 13:30:54 -08:00
Harlow Ward
cd54569c61 Use go-kit Log interface 2015-05-25 22:40:40 -07:00
Harlow Ward
25e390e8b4 Add sample pipeline
* Reduce the size of main README in favor of adding functioning examples
* Add S3 Pipeline exmaple
* Add example of seeing the stream
2015-05-23 15:27:40 -07:00
Harlow Ward
236a6eefd0 Add tags to gitignore 2014-12-13 13:25:36 -08:00
Harlow Ward
06b40e6ed8 Base pipeline components
* Create base Interfaces for Pipeline
* Add first base implementations for Pipeline
* Add initial test for core functionality
2014-11-14 20:45:34 -08:00
Harlow Ward
0fe80b708b Initial commit
* Add installation instructions to README.
* Make note that this repo is under development.
2014-07-25 00:22:06 -07:00