Commit graph

10 commits

Author SHA1 Message Date
Taylor Sutton
f8e9c34641 Refactor zlib decompression into the splitter package
Being in the batchconsumer package means it will work for anything
using KCL, but lambdas that subscribe to these log streams do not use
batchconsumer at all; instead they invoke the splitter package
directly. As such, if we want this functionality to be available to
lambda log consumers, it can't be in batchconsumer.

There are no functionality changes here, just moving code from an
unexported method in one place to an exported function in another
place. The tests also get moved along with it.
2020-11-12 11:37:13 -05:00
Taylor Sutton
6b2d1f8a56 Add zlib decompression to batch consumer.
This has become relevant as zlib is the compression method supported
by the Kinesis plugin for Fluent Bit.
2020-11-11 09:12:05 -05:00
Xavi Ramirez
e5850f1464 Added Initialize method to sender 2018-08-09 16:43:30 -07:00
Rafael Garcia
945ed317c2 remove logger as parameter 2017-11-03 17:48:50 +00:00
Rafael Garcia
221743b3e2 write errors to stderr 2017-11-02 21:50:46 +00:00
Xavi Ramirez
8f7cbf9952 Fixed case checkpoints would only occur on shutdown. The archive consumer fell victim of this bug. 2017-08-20 23:41:28 +00:00
Xavi Ramirez
5c373fa7d8 Fixed unit tests 2017-08-07 19:20:56 +00:00
Xavi Ramirez
6a30e0eb8a Huge refactor. Batchers no longer run in their own go-routines, which allowed us to a lot of concurrency and simplify model. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
ba951ff0da Refactor to fix dead locks and race conditions. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
212ec45d88 Added writter unit tests and fixed a slew of concurrency bugs 2017-07-21 01:35:54 +00:00