This changes fixed cloudwatch metrics publishing by adding long
running go routine to periodically publish cloudwatch metrics.
Also, shutdown metrics publishing when KCL is shutdown.
Test:
Run hmake test and verified cloudwatch metrics has been
published via AWS cloudwatch console.
Jira CNA-702
Change-Id: I78b347cd12939447b0daf93f51acf620d18e2f49
This change enables metrics reporting and fixes a few bug in metrics reporting.
The current metrics reporting is quite limited. Will add more metrics in
next cr.
Tested with both prometheus and cloudwatch.
Jira CNA-702
Change-Id: I678b3f8a372d83f7b8adc419133c14cd10884f61
go languaage doesn't like all-caps on const. Since KCL is mainly from
Amazon's KCL, we'd like the constant to have the exactly same name as
Amazon's KCL. Thefore, skip the lint check.
Change-Id: Ib8a2f52a8f4b44d814eda264f62fdcd53cccc2a7
Add support for handling child/parent shard. When processing
child shard, it has to wait until parent shard finished before
processing itself.
Change-Id: I8bbf104c22ae93409d856be9c6829988c1b2d7eb
This change fixed the bug of not finding checkpoint when process
restart. It also adds missing call to record processor for notifying
the shard info and checkpoint when application first started.
Test:
Run hmake test and verify the log.
Change-Id: I4bdf21ac10c5ee988a0860c140991f7d05975541
This is the core part of KCL by implementing worker.
It has exactly the same interface as Amazon's KCL. Internally,
it uses code from GoKini in order to get the library
functionaly quickly.
This is a working version. The test code worker_test.go
shows how to use this library.
Dynamic resharding feature is out of the scope of M4.
Test:
1. A Kinesis stream named "kcl-test" has been created under photon-infra
account.
2. Download your AWS Credential from IAM user page.
3. Modify the worker_test.go to fill in your aws credential.
4. hmake test
Jira CNA-637
Change-Id: I886d255bab9adaf7a13bca11bfda51bedaacaaed
This is the first part of implementing shard lease for Kinesis
Client library. It creates dynamoDB table for managing
Kinesis stream shard lease.
https://jira.eng.vmware.com/browse/CNA-636
Adjust error code value range.
Change-Id: I16565fa15332843101235fb14545ee69c2599f2f
This is to create configuration and client interface in order to give
user an overview on how the Kinesis client library works.
In order not to reinvent wheel, the api is designed closely aligned with
Amazon Kinesis Client Library in Java.
add errors.
remove @throws and use @error instead.
https://jira.eng.vmware.com/browse/CNA-614
Change-Id: I78a269b328c14df37f878eccef192ff022a669cc
