luciano/amazon-kinesis-client-go
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Amazon Kinesis Client for Go
89 commits 11 branches 1 tag 8.2 MiB
Go 94.4%
Makefile 5.6%
Find a file
Nathan Leiby efa90d4481 splitter: update log stream parsing 
AWS changed the format of the log stream for AWS Batch logs. This
reflects that change.

---

http://docs.aws.amazon.com/batch/latest/userguide/job_states.html

Logs for RUNNING jobs are available in CloudWatch Logs; the log group is /aws/batch/job, and the log stream name format is jobDefinitionName/default/ecs_task_id (this format may change in the future).
After a job reaches the RUNNING status, you can programmatically retrieve its log stream name with the DescribeJobs API operation. For more information, see View Log Data Sent to CloudWatch Logs in the Amazon CloudWatch Logs User Guide. By default, these logs are set to never expire, but you can modify the retention period. For more information, see Change Log Data Retention in CloudWatch Logs in the Amazon CloudWatch Logs User Guide.
2017-09-20 14:18:35 -07:00
batchconsumer derpfix: s/black/blank 2017-09-13 19:22:58 +00:00
cmd (lint) go vet fixes 2017-08-10 14:28:13 -07:00
decode decode: s/type/decoder_msg_type 2017-08-17 14:36:54 -07:00
kcl Standardized error reporting 2017-08-20 23:42:27 +00:00
splitter splitter: update log stream parsing 2017-09-20 14:18:35 -07:00
.gitignore Added vendor to .gitignore 2017-05-22 22:57:34 +00:00
batchconsumer.properties Implemented new consumer interface as well as example consumer: 'batchconsumer' 2017-07-18 02:03:15 +00:00
circle.yml Run benchmarks when building 2017-08-07 19:20:56 +00:00
consumer.properties initial commit 2017-02-08 20:23:00 +00:00
glide.lock Updated syslogparser library and added unit tests 2017-08-16 20:58:30 +00:00
glide.yaml Updated syslogparser library and added unit tests 2017-08-16 20:58:30 +00:00
golang.mk Fixed and added unit tests 2017-07-18 19:19:40 +00:00
Makefile Added less hacky way of downloading jars 2017-08-07 19:22:10 +00:00
README.md initial commit 2017-02-08 20:23:00 +00:00

README.md

amazon-kinesis-client-go

A port of amazon-kinesis-client-python to Go.

Running the Consumer

Edit the file consumer.properties to point at a Kinesis stream that has some data. Build the consumer binary:

make build

Then run:

make run

This will download the jar files necessary to run the KCL, and then launch the KCL communicating with the consumer binary.