Harlow Ward
1b045f6107
Bump sleep time to 10s
...
Noticed GetRecords error when a Shard's lease expired.
* Added a timeout so we don't hit the rate limit on this type of error.
* Increased the timeout when there are no Records on the stream.
2014-11-15 13:51:55 -08:00
Harlow Ward
f921eca908
Add support for JSONPaths in Redshift Load
...
By specifying a `jsonpaths` parameter to an S3 file containing ordinal
attribute position we can store free-form JSON in S3 and then reference
the col order from the COPY command.
http://docs.aws.amazon.com/redshift/latest/dg/r_COPY_command_examples.html
2014-11-15 13:49:37 -08:00
Harlow Ward
06b40e6ed8
Base pipeline components
...
* Create base Interfaces for Pipeline
* Add first base implementations for Pipeline
* Add initial test for core functionality
2014-11-14 20:45:34 -08:00
Harlow Ward
0fe80b708b
Initial commit
...
* Add installation instructions to README.
* Make note that this repo is under development.
2014-07-25 00:22:06 -07:00
