luciano/amazon-kinesis-client-go
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
amazon-kinesis-client-go/splitter
History
Taylor Sutton cc4f7716b3 More documentation around handling of CW log bundles; deflake test
2020-11-16 12:44:50 -05:00
..
README.md Implemented new consumer interface as well as example consumer: 'batchconsumer' 2017-07-18 02:03:15 +00:00
splitter.go More documentation around handling of CW log bundles; deflake test 2020-11-16 12:44:50 -05:00
splitter_test.go More documentation around handling of CW log bundles; deflake test 2020-11-16 12:44:50 -05:00

README.md

splitter

This splitter allows ingesting logs from a CWLogs subscription.

Splitter's expected input is batched logs from a CloudWatchLogs subscription to a Kinesis Stream. The CWLogs subscription has a special format which bundles several logs into a single record. The splitter takes this record and splits it into multiple logs. These logs are also modified to mimic the RSyslog format we expect from our other logs. This allows them to be decoded normally by the rest of the pipeline.