luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
613 commits 48 branches 124 tags 11 MiB
Commit graph

613 commits

This branch
This branch
All branches
Author SHA1 Message Date
ehasah-aws
63911b53a0
Make LAM run at configurable interval (#1464) 
* Make LAM run at configurable interval, tie LeaseDiscoverer with LAM run and tie load balancing with leaseDuration

* remove unwanted SampleApp class

* remove unwanted RecordProcessor and added new parameter class

* Updated comment removed one variable for consistency

* Updated test logic to use Supplier to provide time

* updated logic to count based variance balancing

* Changed variance based balancing to 12

* Changed variance based balancing to 3

* Change logic to balance tied to lease duration

* Change logic to balance tied to LAM run

* Code for backward compatibility

* Code for backward compatibility check

* Code for backward compatibility check

* Best practice to deprecate old constructor

* Best practice to deprecate old constructor

* Best practice to deprecate old constructor

* removed backward compatibility code/constructors

* Formating and remove unused variable

* added formating to avoid build failure
 2025-04-17 10:16:12 -07:00
Minu Hong
d7dd21beca
KCL 3.x documentation update (#1465) 
* Create kcl_3x_deep-dive.md

* Update kcl_3x_deep-dive.md

* Update kcl-configurations.md

 - Add two KCL configurations - WorkerMetricsTableConfig, CoordinatorStateTableConfig - to let users know how to set the custom table name.
 - Update how to set custom names for KCL metadata tables created in DynamoDB
 2025-04-16 10:08:52 -07:00
Abhi Gupta
897d993782
Extending ShardConsumer class constructor to have ConsumerTaskFactory as a param (#1463) 2025-04-14 12:57:58 -07:00
chenylee-aws
6990fc513f
Update Java doc for internal Api annotation (#1466) 2025-04-14 09:32:16 -07:00
skyero-aws
133374706c
Dependabot auto merge addition in github workflows (#1459) 
Dependabot auto-merge feature. Auto-merge triggers for dependabot depencency pull requests that are patches and have a cvss level greater than zero.
 2025-04-01 11:02:17 -07:00
Minu Hong
edd7d9b1e5
Update README.md (#1455) 
* Update README.md

Update KCL main readme page to add end-of-support notice for KCL 1.x. Also applied some formatting changes in the KCL versions section.

* Update README.md

Update the readme

* Update README.md
 2025-03-20 17:37:50 -07:00
lucienlu-aws
458ed653bc
Re-add SNAPSHOT to version (#1453) 2025-03-14 11:12:09 -07:00
vincentvilo-aws
9daed2dda9
Update version number in README for 2.x (#1450) 2025-03-13 10:25:41 -07:00
lucienlu-aws
5263b4227c
Prepare for release 3.0.2 (#1447) 2025-03-12 10:22:41 -07:00
ehasah-aws
7b5ebaeb93
Calculate scan segment by table size (#1443) 
* Fixes DDB usage spike issue

* Removed un-necessary exception handling

* made max total segment size 30

* Cached total scan segment

* Reuse listLeasesParallely api to dynamically calculate total segment

* Added unit tests and made getParallelScanTotalSegments synchronous

* Simplified getParallelScanTotalSegments method

* Fall back to previously calculated totalScan

* fixed formating
 2025-03-12 10:10:19 -07:00
dependabot[bot]
e856e7c95e
Bump io.netty:netty-handler in /amazon-kinesis-client (#1439) 
Bumps [io.netty:netty-handler](https://github.com/netty/netty) from 4.1.108.Final to 4.1.118.Final.
- [Commits](https://github.com/netty/netty/compare/netty-4.1.108.Final...netty-4.1.118.Final)

---
updated-dependencies:
- dependency-name: io.netty:netty-handler
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-03-03 15:17:53 -08:00
Abhi Gupta
c9563ab585
Having a way to override the consumerTaskFactory as part of LeaseManagementConfig (#1441) 2025-02-23 22:17:43 -08:00
Abhi Gupta
8deebe4bda
Adding functionality to override the vanilla KCL tasks (#1440) 2025-02-20 20:26:21 -08:00
ivanlkc
68a7a9bf53
Suppress LeaseAssignmentManager excessive WARN logs when there is no issue (#1437) 
This commit fixes https://github.com/awslabs/amazon-kinesis-client/issues/1407. The WARN level log statements should only be executed when a real problem had been detected.
 2025-02-06 17:20:19 -08:00
Minu Hong
ae9a433ebd
Update README.md (#1434) 
To update the notice related to the issue with AWS Java SDK 2.27.x versions.
 2025-01-27 16:42:54 -08:00
Minu Hong
854d99da52
Update README.md (#1428) 
Update the version number in the dependency for KCL 2.x
 2025-01-03 15:11:04 -08:00
dependabot[bot]
d9a238d33d
Bump com.fasterxml.jackson.core:jackson-databind (#1400) 
Bumps [com.fasterxml.jackson.core:jackson-databind](https://github.com/FasterXML/jackson) from 2.10.1 to 2.12.7.1.
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-databind
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-30 12:25:55 -08:00
vincentvilo-aws
dcac19e927
update dependabot.yml to track master and v2.x branches (#1416) 2024-12-20 10:02:54 -08:00
lucienlu-aws
9da91cd43c
Update version notice to just recommend to use latest version (#1415) 2024-12-18 11:33:06 -08:00
chenylee-aws
4492e1e206
Update version to 3.0.2-SNAPSHOT (#1404) 2024-11-15 12:02:50 -08:00
chenylee-aws
01d0e5cc55
Preparation for v3.0.1 (#1402) 2024-11-14 16:01:36 -08:00
chenylee-aws
3facf303bf
Fix checkpointOwner copy issue for multistream lease (#1401) 2024-11-14 13:50:22 -08:00
Aravinda Kidambi Srinivasan
b154acf7f5
Address KCLv3 issues from github (#1398) 
* Address KCLv3 issues reported on github
1. Fix transitive dependencies and add a maven plugin to catch these
   at build time
2. Remove the redundant shutdown of the leaseCoordinatorThreadPool
3. Fix typo THROUGHOUT_PUT_KBPS
4. Fix shutdown sequence - make sure
    scheduler shutdown without invoking run works
5. Fix backward compatibility check - Avoid flagging methods as deleted
    if it is marked synchronized. Also mark interfaces introduced in KCLv3 as internal.
 2024-11-12 12:20:16 -08:00
Aravinda Kidambi Srinivasan
004f6d3ea0
Update version to 3.0.1-SNAPSHOT (#1394) 
Change version to 3.0.1-SNAPSHOT
 2024-11-07 14:01:51 -08:00
Aravinda Kidambi Srinivasan
f7e909c979
Update backward compatibility check script (#1393) 
Dont fail the check for major release but print the output
for information/review purposes to know what API changes
have occurred in a major release

Manually tested the changes with some incompatible API changes
between patch release, minor release and major release.
 2024-11-06 14:31:25 -08:00
lucienlu-aws
fbfd74df39
Release v3.0.0 (#1392) 2024-11-05 22:09:59 -08:00
dependabot[bot]
a159fa31fb
Bump com.google.protobuf:protobuf-java in /amazon-kinesis-client (#1383) 
Bumps [com.google.protobuf:protobuf-java](https://github.com/protocolbuffers/protobuf) from 4.27.0 to 4.27.5.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-09-30 13:30:44 -07:00
lucienlu-aws
0478575b3d
bugfix: debug or trace logging level now properly logs all worker loops (#1372) 2024-07-26 10:03:46 -07:00
lucienlu-aws
1c0c41c4e8
Add config to enable PITR (#1365) 
* Add config to enable PITR
 2024-07-15 14:15:52 -07:00
nakulj
5878ba8ac6
Add a backwards compatibility check for .proto files (#1362) 2024-07-12 11:18:54 -07:00
nakulj
3b5c59ec04
fix backwards compatibility check (#1361) 2024-07-12 09:46:49 -07:00
nakulj
57dcddf10b
Fix some of the warnings emitted by maven during build (#1363) 
* Set the source encoding in the pom.

This stops the following warning from being emitted in our logs:

```
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
```

* Replace deprecated systemProperties.

The recommended replacement is systemPropertyVariables, documented [here](https://maven.apache.org/surefire/maven-surefire-plugin/examples/system-properties.html).
 2024-07-08 15:22:24 -07:00
Nakul Joshi
f507066ea6 *Actually* skip tests while running backwards compatibility test. 2024-07-02 14:00:57 -07:00
Aravinda Kidambi Srinivasan
6cba7f431d
Remove ShutdownNotificationAware and update javadocs (#1358) 
* Deprecate ShutdownNotificationAware and update javadocs

ShutdownNotificationAware is not used by KCL, this PR
marks it as deprecated and updates the javadoc

Co-authored-by: nakulj <nklj@amazon.com>
 2024-07-02 13:28:19 -07:00
lucienlu-aws
715690d2c0
bugfix: revert update of in memory lease if DDB lease renewal throws error (#1354) 2024-06-24 10:47:00 -07:00
dependabot[bot]
78fb42ede1 Bump commons-io:commons-io from 2.15.1 to 2.16.1 
Bumps commons-io:commons-io from 2.15.1 to 2.16.1.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-07 11:06:32 -07:00
dependabot[bot]
b271fed18b Bump awssdk.version from 2.25.11 to 2.25.64 
Bumps `awssdk.version` from 2.25.11 to 2.25.64.

Updates `software.amazon.awssdk:kinesis` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:dynamodb` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:cloudwatch` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:netty-nio-client` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:sts` from 2.25.11 to 2.25.64

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-07 11:05:55 -07:00
dependabot[bot]
cbed8c3c6f Bump com.google.protobuf:protobuf-java from 3.21.12 to 4.27.0 
Bumps [com.google.protobuf:protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.12 to 4.27.0.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-04 10:04:19 -07:00
nakulj
e8c3c01b12
Generate wrappers from proto files instead of shipping them directly (#1340) 
* Replace generated code with source .proto files

This change replaces the generated Messages.java file with the source .proto files.
It also includes the pom.xml changes needed to install protoc from Maven Central, and use it to compile the wrapper java code.

* add explanatory comment
 2024-06-03 15:20:07 -07:00
dependabot[bot]
a83b3fbd57 Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.6.3 to 3.7.0 
Bumps [org.apache.maven.plugins:maven-javadoc-plugin](https://github.com/apache/maven-javadoc-plugin) from 3.6.3 to 3.7.0.
- [Release notes](https://github.com/apache/maven-javadoc-plugin/releases)
- [Commits](https://github.com/apache/maven-javadoc-plugin/compare/maven-javadoc-plugin-3.6.3...maven-javadoc-plugin-3.7.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-javadoc-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-03 09:06:16 -07:00
dependabot[bot]
0005670513 Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.5 
Bumps [org.apache.maven.plugins:maven-failsafe-plugin](https://github.com/apache/maven-surefire) from 3.1.2 to 3.2.5.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.1.2...surefire-3.2.5)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-failsafe-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-03 09:05:50 -07:00
dependabot[bot]
35b150009a Bump org.slf4j:slf4j-api from 2.0.7 to 2.0.13 
Bumps org.slf4j:slf4j-api from 2.0.7 to 2.0.13.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-31 13:18:54 -07:00
dependabot[bot]
d3b84dc995 Bump io.reactivex.rxjava3:rxjava from 3.1.6 to 3.1.8 
Bumps [io.reactivex.rxjava3:rxjava](https://github.com/ReactiveX/RxJava) from 3.1.6 to 3.1.8.
- [Release notes](https://github.com/ReactiveX/RxJava/releases)
- [Commits](https://github.com/ReactiveX/RxJava/compare/v3.1.6...v3.1.8)

---
updated-dependencies:
- dependency-name: io.reactivex.rxjava3:rxjava
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-31 12:56:25 -07:00
Nakul Joshi
49fa32d68e Update maven compiler plugin to 3.13.0 
This version of the compiler plugin supports the `--release` option as
documented [here](https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-release.html).
 2024-05-30 09:33:41 -07:00
Nakul Joshi
221db0564d Fix checkstyle violations. 
Follow up to #1332.
- Manually split up a long string literal.
- Disable ArrayTrailingComma, which checkstyle complains about even in
  commented code. In any case it is redundant with PJF.
 2024-05-21 16:35:31 -07:00
Nakul Joshi
adcc89f80d ignore format commit in blame 2024-05-21 01:27:20 -07:00
Nakul Joshi
18fe49eed0 run spotless:apply 2024-05-21 01:27:20 -07:00
Nakul Joshi
409a8f13cd add spotless 2024-05-21 01:27:20 -07:00
Nakul Joshi
c5e4fa2de0 remove commented imports 2024-05-21 01:27:20 -07:00
dependabot[bot]
a8abbea29f Bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.5 
Bumps [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) from 3.1.2 to 3.2.5.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.1.2...surefire-3.2.5)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-20 16:21:00 -07:00