luciano/vmware-go-kcl-v2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
vmware-go-kcl is a vmware originated open-source project for AWS Kinesis Client Library in Go. It has been widely used by many external companies and internally by Carbon Black. vmware-go-kcl-v2 is its companion project by utilizing AWS Go SDK V2 which introduces lots of breaking changes. To keep the repo clean, it is better to have a separated repo vmware-go-kcl-v2 with better golang project structure improvement.
49 commits 18 branches 1 tag 598 KiB
Go 97%
Shell 2.3%
Makefile 0.7%
Find a file
Tao Jiang 0d91fbd443 Add generic logger support (#43) 
* Add generic logger support

The current KCL has tight coupling with logrus and it causes
issue for customer to use different logging system such as zap log.
The issue has been opened via:
https://github.com/vmware/vmware-go-kcl/issues/27

This change is to created a logger interface be able to abstract
above logrus and zap log. It makes easy to add support for other
logging system in the fugure. The work is based on:
https://www.mountedthoughts.com/golang-logger-interface/

Some updates are made in order to make logging system easily
injectable and add more unit tests.

Tested against real kinesis and dyamodb as well.

Signed-off-by: Tao Jiang <taoj@vmware.com>

* Add lumberjack configuration options to have fine grained control

Update the file log configuratio by adding most of luberjack
configuration to avoid hardcode default value. Let user to specify
the value because log retention and rotation are very important
for prod environment.

Signed-off-by: Tao Jiang <taoj@vmware.com>
2021-12-20 21:21:14 -06:00
clientlibrary Add generic logger support (#43) 2021-12-20 21:21:14 -06:00
logger Add generic logger support (#43) 2021-12-20 21:21:14 -06:00
support Release shard lease after shutdown (#31) 2021-12-20 21:21:14 -06:00
test Add generic logger support (#43) 2021-12-20 21:21:14 -06:00
.gitignore CKL: Initial version for starting repository 2021-12-20 21:13:11 -06:00
.gitreview CKL: Initial version for starting repository 2021-12-20 21:13:11 -06:00
CODE_OF_CONDUCT.md Add Code of Conduct template 2021-11-30 23:05:17 +08:00
CONTRIBUTING.md Add DCO text 2021-12-20 21:20:10 -06:00
go.mod Add generic logger support (#43) 2021-12-20 21:21:14 -06:00
go.sum Add generic logger support (#43) 2021-12-20 21:21:14 -06:00
HyperMake Use AWS built-in retry logic and refactor tests (#24) 2021-12-20 21:21:14 -06:00
LICENSE KCL: Update copyright and permission 2021-12-20 21:16:27 -06:00
README.md Add DCO text 2021-12-20 21:19:26 -06:00

README.md

vmware-go-kcl-v2

Overview

Try it out

Prerequisites

  • Prereq 1
  • Prereq 2
  • Prereq 3

Build & Run

  1. Step 1
  2. Step 2
  3. Step 3

Documentation

Contributing

The vmware-go-kcl-v2 project team welcomes contributions from the community. Before you start working with vmware-go-kcl-v2, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.

License

Build & Run

hmake

# security scan
hmake scanast

# run test
hmake check

# run integration test
# update the worker_test.go to let it point to your Kinesis stream
hmake test

Documentation

VMware-Go-KCL matches exactly the same interface and programming model from original Amazon KCL, the best place for getting reference, tutorial is from Amazon itself:

Contributing

The vmware-go-kcl project team welcomes contributions from the community. Before you start working with vmware-go-kcl, please read our Developer Certificate of Origin. All contributions to this repository must be signed as described on that page. Your signature certifies that you wrote the patch or have the right to pass it on as an open-source patch. For more detailed information, refer to CONTRIBUTING.md.

License

MIT License