luciano/amazon-kinesis-client-go
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
50 commits 11 branches 1 tag 8.2 MiB
Commit graph

50 commits

This branch
This branch
All branches
Author SHA1 Message Date
Xavi Ramirez
4809cdb4e6 Another refactor to simply and likely speed up code. New code allows checkpoint and readline messages to be interleaved. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
1bff01ff4f Removed unused params and properties 2017-08-07 19:20:56 +00:00
Xavi Ramirez
734b46274f Better coordinated checkpoints and readlines. Checkpoint function now returns all errors. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
b19c9297d8 Adjusted some locks 2017-08-07 19:20:56 +00:00
Xavi Ramirez
c814742afa Another refactor 2017-08-07 19:20:56 +00:00
Xavi Ramirez
c5f75d6554 Fixed another race-condition. Ensure line is completely processed before allowing a checkpoint. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
6a30e0eb8a Huge refactor. Batchers no longer run in their own go-routines, which allowed us to a lot of concurrency and simplify model. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
ba951ff0da Refactor to fix dead locks and race conditions. 2017-08-07 19:20:56 +00:00
Xavi Ramirez
29f68f77eb Removed redundant retry logic 2017-08-07 19:20:56 +00:00
Xavi Ramirez
c6fe4cef37 Renamed variable from seq to pair 2017-08-07 19:20:56 +00:00
Xavi Ramirez
6e9457cbcf Better propogated shutdown signal 2017-08-07 19:20:56 +00:00
Xavi Ramirez
873544ae78 Run benchmarks when building 2017-08-07 19:20:56 +00:00
Xavi Ramirez
266d7d620d Fixed bug in decoder package 2017-08-07 19:20:56 +00:00
Xavi Ramirez
7a7716e824 Increased default read rate limit 2017-08-07 19:20:56 +00:00
Xavi Ramirez
04042290f5 Moved SequencePair to kcl package 2017-08-07 19:20:56 +00:00
Xavi
4c6d8453ef Merge pull request #5 from Clever/INFRA-2405-new-consumer-interface 
Implemented new consumer interface as well as example consumer
 2017-07-24 13:12:43 -07:00
Xavi Ramirez
212ec45d88 Added writter unit tests and fixed a slew of concurrency bugs 2017-07-21 01:35:54 +00:00
Xavi Ramirez
939fc5406f Batcher now returns error if bad config is provided 2017-07-21 01:33:59 +00:00
Xavi Ramirez
0cb14948a3 KCL more aggressively logs errors 2017-07-21 01:27:17 +00:00
Xavi Ramirez
ae37b57e61 Added IsEmpty to SequencePair 2017-07-21 01:25:02 +00:00
Xavi Ramirez
b0f769bfa7 Breaking change to KCL. Created Checkpointer interface to make testing easier. 2017-07-21 01:25:02 +00:00
Xavi Ramirez
164b9923be Reworded docs to remove references to firehose. Renamed config variables to be clearer. 2017-07-21 01:25:02 +00:00
Xavi Ramirez
e8f40e607a Added unit test for SequencePair IsLessThan function 2017-07-19 19:00:28 +00:00
Xavi Ramirez
1b2afcafc0 Renamed methods to be more generic and to remove reference to logs 2017-07-19 00:21:31 +00:00
Xavi Ramirez
f116c752f5 Added unit test to ensure correct error is returned 2017-07-19 00:20:20 +00:00
Xavi Ramirez
906f6b94fc Made ReadBurstLimit relative to read rate limit 2017-07-18 19:53:25 +00:00
Xavi Ramirez
3f42cb5e4a Created IsLessThan method to SequencePair to make code more readable 2017-07-18 19:52:26 +00:00
Xavi Ramirez
6d4df426a9 Added install deps step to circle yaml 2017-07-18 19:31:06 +00:00
Xavi Ramirez
8d273d6a1e Fixed and added unit tests 2017-07-18 19:19:40 +00:00
Xavi Ramirez
a329c40645 Fixed race condition 2017-07-18 19:13:39 +00:00
Xavi Ramirez
8bcef6cd91 Log file closed when kclProcessor is done running 2017-07-18 19:12:46 +00:00
Xavi Ramirez
4c67f39c99 Implemented new consumer interface as well as example consumer: 'batchconsumer' 2017-07-18 02:03:15 +00:00
Nathan Leiby
aef51b0801 Merge pull request #4 from Clever/use-golang-1.8 
Go 1.7 -> 1.8
 2017-05-25 15:29:33 -07:00
Nathan Leiby
bb9ebbca1d Use Go 1.8 in Circle build and golang-version-check 2017-05-24 12:40:55 -07:00
Xavi
92eaacfbf1 Merge pull request #3 from Clever/made-checkpointer-cacheable 
Consumers of this library can now cache the checkpointer object
 2017-05-22 16:49:23 -07:00
Xavi Ramirez
458e66e321 Added shutdown method to *Checkerpointer struct 2017-05-22 23:06:56 +00:00
Xavi Ramirez
5f507ab116 Moved retry and error handling logic to Checkpointer class 2017-05-22 23:06:16 +00:00
Xavi Ramirez
02b053c0ee Added vendor to .gitignore 2017-05-22 22:57:34 +00:00
Xavi Ramirez
3c56b57e6b Added mutex to Checkpoint function to avoid racey conditions 2017-05-19 22:30:11 +00:00
Xavi Ramirez
17cfe98efa Users of this library can now cache the checkpointer object 2017-05-19 22:01:57 +00:00
Rafael
8da04c944f Merge pull request #2 from Clever/shutdown-checkpoint 
allow for empty checkpoints on shutdown
 2017-05-17 17:52:42 -07:00
Rafael Garcia
bea2abc691 allow for empty checkpoints on shutdown 2017-05-18 00:46:29 +00:00
Nathan Leiby
695f85d99d Merge pull request #1 from Clever/checkpointing-fix 
checkpointing fix
 2017-04-05 15:05:39 -07:00
Nathan Leiby
e6229ce8bb checkpointing fix 
- always try checkpoint at least once (previously, a value of 0 retries
  meant it would never checkpoint)
- setup record processor using New(), so we get default checkpointing
  configuration
 2017-04-05 13:51:45 -07:00
Rafael Garcia
47eec92a1a more circle tweaks 2017-02-08 20:38:14 +00:00
Rafael Garcia
cd0fa0ab62 circle tweaks 2017-02-08 20:35:07 +00:00
Rafael Garcia
0685d1e90a circle.yml 2017-02-08 20:29:23 +00:00
Rafael Garcia
14568eb5ad don't need glide 2017-02-08 20:25:43 +00:00
Rafael Garcia
39e5fa5860 initial commit 2017-02-08 20:23:00 +00:00
clever-init-service
69414be395 Initial commit 2017-02-08 12:03:06 -08:00