Xavi Ramirez
|
e2f55b6d98
|
Updated syslogparser library and added unit tests
|
2017-08-16 20:58:30 +00:00 |
|
Xavi Ramirez
|
55aeecddd7
|
Renamed IsEmpty to IsNil on SequencePair
|
2017-08-10 20:16:41 +00:00 |
|
Xavi Ramirez
|
45fad863d0
|
Made config object that's specific to BatcherManager
|
2017-08-10 20:11:24 +00:00 |
|
Xavi Ramirez
|
3a04dad4bf
|
Send single value instead of full config object to CheckpointManager
|
2017-08-10 20:01:07 +00:00 |
|
Xavi Ramirez
|
6102f9c752
|
Added comment to why a channel is taking a channel.
|
2017-08-10 19:56:17 +00:00 |
|
Xavi Ramirez
|
68a16cfe6a
|
Added an external go routine to ensure stale batches get flushed
|
2017-08-10 19:47:08 +00:00 |
|
Xavi Ramirez
|
27cce44140
|
Made constructors of private structs private
|
2017-08-10 19:21:46 +00:00 |
|
Xavi Ramirez
|
1632c81fc7
|
Use errors.New instead of fmt.Errorf
|
2017-08-10 19:20:55 +00:00 |
|
Xavi Ramirez
|
dd9da85055
|
Fixed type-o
|
2017-08-10 19:15:53 +00:00 |
|
Xavi Ramirez
|
b7743c9ea7
|
Added a more coherent shutdown pathway
|
2017-08-09 01:22:11 +00:00 |
|
Xavi Ramirez
|
1be812a887
|
Added less hacky way of downloading jars
|
2017-08-07 19:22:10 +00:00 |
|
Xavi Ramirez
|
de04a27799
|
Return NonKayveeError if log contians invalid or empty json
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
eb230b94f7
|
Added stats to base kinesis client
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
5c373fa7d8
|
Fixed unit tests
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
4809cdb4e6
|
Another refactor to simply and likely speed up code. New code allows checkpoint and readline messages to be interleaved.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
1bff01ff4f
|
Removed unused params and properties
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
734b46274f
|
Better coordinated checkpoints and readlines. Checkpoint function now returns all errors.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
b19c9297d8
|
Adjusted some locks
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
c814742afa
|
Another refactor
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
c5f75d6554
|
Fixed another race-condition. Ensure line is completely processed before allowing a checkpoint.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
6a30e0eb8a
|
Huge refactor. Batchers no longer run in their own go-routines, which allowed us to a lot of concurrency and simplify model.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
ba951ff0da
|
Refactor to fix dead locks and race conditions.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
29f68f77eb
|
Removed redundant retry logic
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
c6fe4cef37
|
Renamed variable from seq to pair
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
6e9457cbcf
|
Better propogated shutdown signal
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
873544ae78
|
Run benchmarks when building
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
266d7d620d
|
Fixed bug in decoder package
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
7a7716e824
|
Increased default read rate limit
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
04042290f5
|
Moved SequencePair to kcl package
|
2017-08-07 19:20:56 +00:00 |
|
Xavi
|
4c6d8453ef
|
Merge pull request #5 from Clever/INFRA-2405-new-consumer-interface
Implemented new consumer interface as well as example consumer
|
2017-07-24 13:12:43 -07:00 |
|
Xavi Ramirez
|
212ec45d88
|
Added writter unit tests and fixed a slew of concurrency bugs
|
2017-07-21 01:35:54 +00:00 |
|
Xavi Ramirez
|
939fc5406f
|
Batcher now returns error if bad config is provided
|
2017-07-21 01:33:59 +00:00 |
|
Xavi Ramirez
|
0cb14948a3
|
KCL more aggressively logs errors
|
2017-07-21 01:27:17 +00:00 |
|
Xavi Ramirez
|
ae37b57e61
|
Added IsEmpty to SequencePair
|
2017-07-21 01:25:02 +00:00 |
|
Xavi Ramirez
|
b0f769bfa7
|
Breaking change to KCL. Created Checkpointer interface to make testing easier.
|
2017-07-21 01:25:02 +00:00 |
|
Xavi Ramirez
|
164b9923be
|
Reworded docs to remove references to firehose. Renamed config variables to be clearer.
|
2017-07-21 01:25:02 +00:00 |
|
Xavi Ramirez
|
e8f40e607a
|
Added unit test for SequencePair IsLessThan function
|
2017-07-19 19:00:28 +00:00 |
|
Xavi Ramirez
|
1b2afcafc0
|
Renamed methods to be more generic and to remove reference to logs
|
2017-07-19 00:21:31 +00:00 |
|
Xavi Ramirez
|
f116c752f5
|
Added unit test to ensure correct error is returned
|
2017-07-19 00:20:20 +00:00 |
|
Xavi Ramirez
|
906f6b94fc
|
Made ReadBurstLimit relative to read rate limit
|
2017-07-18 19:53:25 +00:00 |
|
Xavi Ramirez
|
3f42cb5e4a
|
Created IsLessThan method to SequencePair to make code more readable
|
2017-07-18 19:52:26 +00:00 |
|
Xavi Ramirez
|
6d4df426a9
|
Added install deps step to circle yaml
|
2017-07-18 19:31:06 +00:00 |
|
Xavi Ramirez
|
8d273d6a1e
|
Fixed and added unit tests
|
2017-07-18 19:19:40 +00:00 |
|
Xavi Ramirez
|
a329c40645
|
Fixed race condition
|
2017-07-18 19:13:39 +00:00 |
|
Xavi Ramirez
|
8bcef6cd91
|
Log file closed when kclProcessor is done running
|
2017-07-18 19:12:46 +00:00 |
|
Xavi Ramirez
|
4c67f39c99
|
Implemented new consumer interface as well as example consumer: 'batchconsumer'
|
2017-07-18 02:03:15 +00:00 |
|
Nathan Leiby
|
aef51b0801
|
Merge pull request #4 from Clever/use-golang-1.8
Go 1.7 -> 1.8
|
2017-05-25 15:29:33 -07:00 |
|
Nathan Leiby
|
bb9ebbca1d
|
Use Go 1.8 in Circle build and golang-version-check
|
2017-05-24 12:40:55 -07:00 |
|
Xavi
|
92eaacfbf1
|
Merge pull request #3 from Clever/made-checkpointer-cacheable
Consumers of this library can now cache the checkpointer object
|
2017-05-22 16:49:23 -07:00 |
|
Xavi Ramirez
|
458e66e321
|
Added shutdown method to *Checkerpointer struct
|
2017-05-22 23:06:56 +00:00 |
|