luciano/kinesis-consumer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update link for AWS attribution

Browse source
This commit is contained in:
Harlow Ward 2014-11-15 18:10:19 -08:00
parent a9a625aa79
commit 02a7c648a3
1 changed files with 4 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
README.md
View file

@ -4,8 +4,8 @@ __Note:__ _This codebase is a under active development._
### Kinesis connector applications written in Go ### Kinesis connector applications written in Go
This is a port of the [AWS Kinesis connector libraries][2] from Java to Go for extracting streaming event data This is a port of the [Amazon Kinesis Connector Library][1] from Java to Go. Its used for extracting streaming event data
into S3, Redshift, DynamoDB, and more. See the [API Docs][1] for package documentation. into S3, Redshift, DynamoDB, and more. See the [API Docs][2] for package documentation.
## Overview ## Overview
@ -27,8 +27,6 @@ Install the library:
$ go get github.com/harlow/kinesis-connectors $ go get github.com/harlow/kinesis-connectors
The library has been broken into several components (buffers, checkpoints, filters, transformers, and emitters). These compontents can be mixed and matched to generate the desired functionality.
### Example Redshift Pipeline ### Example Redshift Pipeline
The Redshift Pipeline will pull records from Kinesis and buffer them untill the desired threshold is reached. The Emitter will then upload the buffered records to an S3 bucket, set a checkpoint in Redis, and copy data to to Redshift. The Redshift Pipeline will pull records from Kinesis and buffer them untill the desired threshold is reached. The Emitter will then upload the buffered records to an S3 bucket, set a checkpoint in Redis, and copy data to to Redshift.
@ -131,6 +129,5 @@ func main() {
} }
``` ```
[1]: http://godoc.org/github.com/harlow/kinesis-connectors [1]: https://github.com/awslabs/amazon-kinesis-connectors
[2]: http://aws.amazon.com/kinesis/ [2]: http://godoc.org/github.com/harlow/kinesis-connectors
[3]: https://github.com/awslabs/amazon-kinesis-connectors