Harlow Ward
f0e6461cb6
Refactor to use handler func
...
The previous pipeline model required a lot of setup and abstracted away
the processing of records. By passing a HandlerFunc to the consumer we
keep the business logic of processing of records closer to the use of
the consumer.
* Add refactoring note and SHA to README
2016-02-06 17:50:17 -08:00
Harlow Ward
3d9e6e2485
Add Redshift Pipeline example
2015-05-25 23:18:14 -07:00
Harlow Ward
e17988cfc9
Update Logging instructions
2015-05-25 23:09:59 -07:00
Harlow Ward
a27a13f405
Update README.md
2015-05-23 16:18:10 -07:00
Harlow Ward
9a30e97f10
Add link to exmaples directory
2015-05-23 16:10:08 -07:00
Harlow Ward
25e390e8b4
Add sample pipeline
...
* Reduce the size of main README in favor of adding functioning examples
* Add S3 Pipeline exmaple
* Add example of seeing the stream
2015-05-23 15:27:40 -07:00
Harlow Ward
fdd39482bb
Add Installation sub-header to Usage
2015-05-23 13:57:52 -07:00
Harlow Ward
b9a489b940
Update header size. Remove note
2015-05-23 13:56:03 -07:00
Harlow Ward
6ca2abb7f8
User Log package for default logger
...
* Modify the default logger to it also logs Line Numbers
* Update README w/ logging instructions
2015-05-23 13:52:57 -07:00
Harlow Ward
9077d288d2
Rename License file and add Contributing sections
...
* Make it clear from project root we are using MIT-LICENSE
* Add contributing section w/ overview of how to submit code to repo
2015-05-23 10:24:53 -07:00
Harlow Ward
bde3e96ad9
Add Discard Logger
...
For situations where we don't want any logs produced by the application.
* Remove references to Logger from README
* Add implementation of DiscardLogger
2015-05-22 23:38:06 -07:00
Harlow Ward
82d10ab78d
Simplify example (no Redshift cluster required)
2015-05-03 21:01:21 -07:00
Harlow Ward
cd71fd41bc
Add Logger interface
...
To allow for different logging endpoints we'll introduce a Logger
interface that will be passed into the pipeline during initialization.
* Add Logger interface
* Use logger interface in pipeline and emitters
2015-05-03 21:01:14 -07:00
Harlow Ward
da7a1dd87a
Update README.md
2014-12-20 19:46:38 -08:00
Harlow Ward
1bce60bae6
Update example with Redshift Manifest Pipeline
2014-12-20 19:45:34 -08:00
Harlow Ward
05920cc5d3
Update development notice
2014-12-18 18:26:59 -08:00
Harlow Ward
106f0d66bb
Add Redshift Manifest functionality
...
Use a manifest file for batch importing files uploaded to S3.
2014-12-10 21:59:42 -08:00
Harlow Ward
02a7c648a3
Update link for AWS attribution
2014-11-15 18:10:19 -08:00
Harlow Ward
a9a625aa79
Update example usage in README
...
* Showcase the new StringToStringTransformer as a simple way of getting
data directly from Kinesis to Redshift.
2014-11-15 17:36:28 -08:00
Harlow Ward
06b40e6ed8
Base pipeline components
...
* Create base Interfaces for Pipeline
* Add first base implementations for Pipeline
* Add initial test for core functionality
2014-11-14 20:45:34 -08:00
Harlow Ward
0fe80b708b
Initial commit
...
* Add installation instructions to README.
* Make note that this repo is under development.
2014-07-25 00:22:06 -07:00
