Rafael Garcia
|
7b18d1bab4
|
write failed log message to file
|
2017-11-03 18:03:39 +00:00 |
|
Rafael Garcia
|
945ed317c2
|
remove logger as parameter
|
2017-11-03 17:48:50 +00:00 |
|
Rafael Garcia
|
221743b3e2
|
write errors to stderr
|
2017-11-02 21:50:46 +00:00 |
|
mohit
|
22495acfe4
|
derpfix: s/black/blank
|
2017-09-13 19:22:58 +00:00 |
|
mohit
|
5b91cd80da
|
DefaultCounters to log 0 for counter stats
this makes sure that we can get metrics when things are not flowing
through one of the consumers
|
2017-09-12 04:33:05 +00:00 |
|
mohit
|
8b44537934
|
fix typo in error log
|
2017-09-11 20:22:34 +00:00 |
|
Xavi Ramirez
|
f581bb4f6e
|
Increased default read rate.
|
2017-08-22 18:37:47 +00:00 |
|
Xavi Ramirez
|
8f7cbf9952
|
Fixed case checkpoints would only occur on shutdown. The archive consumer fell victim of this bug.
|
2017-08-20 23:41:28 +00:00 |
|
Nathan Leiby
|
6039f69d86
|
(lint) go vet fixes
|
2017-08-10 14:28:13 -07:00 |
|
Nathan Leiby
|
5b8a86212d
|
remove DeployEnv (no longer needed)
|
2017-08-10 14:26:27 -07:00 |
|
Xavi Ramirez
|
55aeecddd7
|
Renamed IsEmpty to IsNil on SequencePair
|
2017-08-10 20:16:41 +00:00 |
|
Xavi Ramirez
|
45fad863d0
|
Made config object that's specific to BatcherManager
|
2017-08-10 20:11:24 +00:00 |
|
Xavi Ramirez
|
3a04dad4bf
|
Send single value instead of full config object to CheckpointManager
|
2017-08-10 20:01:07 +00:00 |
|
Xavi Ramirez
|
6102f9c752
|
Added comment to why a channel is taking a channel.
|
2017-08-10 19:56:17 +00:00 |
|
Xavi Ramirez
|
68a16cfe6a
|
Added an external go routine to ensure stale batches get flushed
|
2017-08-10 19:47:08 +00:00 |
|
Xavi Ramirez
|
27cce44140
|
Made constructors of private structs private
|
2017-08-10 19:21:46 +00:00 |
|
Xavi Ramirez
|
1632c81fc7
|
Use errors.New instead of fmt.Errorf
|
2017-08-10 19:20:55 +00:00 |
|
Xavi Ramirez
|
b7743c9ea7
|
Added a more coherent shutdown pathway
|
2017-08-09 01:22:11 +00:00 |
|
Xavi Ramirez
|
eb230b94f7
|
Added stats to base kinesis client
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
5c373fa7d8
|
Fixed unit tests
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
734b46274f
|
Better coordinated checkpoints and readlines. Checkpoint function now returns all errors.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
6a30e0eb8a
|
Huge refactor. Batchers no longer run in their own go-routines, which allowed us to a lot of concurrency and simplify model.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
ba951ff0da
|
Refactor to fix dead locks and race conditions.
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
29f68f77eb
|
Removed redundant retry logic
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
c6fe4cef37
|
Renamed variable from seq to pair
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
6e9457cbcf
|
Better propogated shutdown signal
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
7a7716e824
|
Increased default read rate limit
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
04042290f5
|
Moved SequencePair to kcl package
|
2017-08-07 19:20:56 +00:00 |
|
Xavi Ramirez
|
212ec45d88
|
Added writter unit tests and fixed a slew of concurrency bugs
|
2017-07-21 01:35:54 +00:00 |
|
Xavi Ramirez
|
939fc5406f
|
Batcher now returns error if bad config is provided
|
2017-07-21 01:33:59 +00:00 |
|
Xavi Ramirez
|
ae37b57e61
|
Added IsEmpty to SequencePair
|
2017-07-21 01:25:02 +00:00 |
|
Xavi Ramirez
|
164b9923be
|
Reworded docs to remove references to firehose. Renamed config variables to be clearer.
|
2017-07-21 01:25:02 +00:00 |
|
Xavi Ramirez
|
e8f40e607a
|
Added unit test for SequencePair IsLessThan function
|
2017-07-19 19:00:28 +00:00 |
|
Xavi Ramirez
|
1b2afcafc0
|
Renamed methods to be more generic and to remove reference to logs
|
2017-07-19 00:21:31 +00:00 |
|
Xavi Ramirez
|
f116c752f5
|
Added unit test to ensure correct error is returned
|
2017-07-19 00:20:20 +00:00 |
|
Xavi Ramirez
|
906f6b94fc
|
Made ReadBurstLimit relative to read rate limit
|
2017-07-18 19:53:25 +00:00 |
|
Xavi Ramirez
|
3f42cb5e4a
|
Created IsLessThan method to SequencePair to make code more readable
|
2017-07-18 19:52:26 +00:00 |
|
Xavi Ramirez
|
8d273d6a1e
|
Fixed and added unit tests
|
2017-07-18 19:19:40 +00:00 |
|
Xavi Ramirez
|
a329c40645
|
Fixed race condition
|
2017-07-18 19:13:39 +00:00 |
|
Xavi Ramirez
|
8bcef6cd91
|
Log file closed when kclProcessor is done running
|
2017-07-18 19:12:46 +00:00 |
|
Xavi Ramirez
|
4c67f39c99
|
Implemented new consumer interface as well as example consumer: 'batchconsumer'
|
2017-07-18 02:03:15 +00:00 |
|