luciano/kinesis-consumer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source
This commit is contained in:
Harlow Ward 2015-05-23 16:18:10 -07:00
parent 9a30e97f10
commit a27a13f405
1 changed files with 20 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
README.md
View file

@ -1,6 +1,6 @@
# Golang Kinesis Connectors # Golang Kinesis Connectors
#### Kinesis connector applications written in Go __Kinesis connector applications written in Go__
Inspired by the [Amazon Kinesis Connector Library][1]. These components are used for extracting streaming event data Inspired by the [Amazon Kinesis Connector Library][1]. These components are used for extracting streaming event data
into S3, Redshift, DynamoDB, and more. See the [API Docs][2] for package documentation. into S3, Redshift, DynamoDB, and more. See the [API Docs][2] for package documentation.
@ -8,12 +8,12 @@ into S3, Redshift, DynamoDB, and more. See the [API Docs][2] for package documen
[1]: https://github.com/awslabs/amazon-kinesis-connectors [1]: https://github.com/awslabs/amazon-kinesis-connectors
[2]: http://godoc.org/github.com/harlow/kinesis-connectors [2]: http://godoc.org/github.com/harlow/kinesis-connectors
![golang_kinesis_connector](https://cloud.githubusercontent.com/assets/739782/4262283/2ee2550e-3b97-11e4-8cd1-21a5d7ee0964.png)
## Overview ## Overview
Each Amazon Kinesis connector application is a pipeline that determines how records from an Amazon Kinesis stream will be handled. Records are retrieved from the stream, transformed according to a user-defined data model, buffered for batch processing, and then emitted to the appropriate AWS service. Each Amazon Kinesis connector application is a pipeline that determines how records from an Amazon Kinesis stream will be handled. Records are retrieved from the stream, transformed according to a user-defined data model, buffered for batch processing, and then emitted to the appropriate AWS service.
![golang_kinesis_connector](https://cloud.githubusercontent.com/assets/739782/4262283/2ee2550e-3b97-11e4-8cd1-21a5d7ee0964.png)
A connector pipeline uses the following interfaces: A connector pipeline uses the following interfaces:
* __Pipeline:__ The pipeline implementation itself. * __Pipeline:__ The pipeline implementation itself.
@ -28,13 +28,24 @@ Get the package source:
$ go get github.com/harlow/kinesis-connectors $ go get github.com/harlow/kinesis-connectors
### Examples Import the `go-kinesis` and `kinesis-connector` packages:
Examples pipelines are proviede in [examples directory][demo]. ```go
package main
[demo]: https://github.com/harlow/kinesis-connectors/tree/master/examples import (
"github.com/harlow/kinesis-connectors"
"github.com/sendgridlabs/go-kinesis"
)
```
### Custom Logging ### Example Pipelines
Examples pipelines are proviede in [examples directory][example].
[example]: https://github.com/harlow/kinesis-connectors/tree/master/examples
### Logger
Default logging is handled by Package log. An application can override the defualt package logging by Default logging is handled by Package log. An application can override the defualt package logging by
changing it's `logger` variable: changing it's `logger` variable:
@ -47,14 +58,14 @@ The customer logger must implement the [Logger interface][log_interface].
[log_interface]: https://github.com/harlow/kinesis-connectors/blob/master/logger.go [log_interface]: https://github.com/harlow/kinesis-connectors/blob/master/logger.go
### Contributing ## Contributing
Please see [CONTRIBUTING.md] for more information. Thank you, [contributors]! Please see [CONTRIBUTING.md] for more information. Thank you, [contributors]!
[LICENSE]: /MIT-LICENSE [LICENSE]: /MIT-LICENSE
[CONTRIBUTING.md]: /CONTRIBUTING.md [CONTRIBUTING.md]: /CONTRIBUTING.md
### License ## License
Copyright (c) 2015 Harlow Ward. It is free software, and may Copyright (c) 2015 Harlow Ward. It is free software, and may
be redistributed under the terms specified in the [LICENSE] file. be redistributed under the terms specified in the [LICENSE] file.