luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
599 commits 48 branches 124 tags 11 MiB
Commit graph

599 commits

This branch
This branch
All branches
Author SHA1 Message Date
Minu Hong
ddb7b89c6a
Update README.md 
To update the notice related to the issue with AWS Java SDK 2.27.x versions.
 2025-01-27 15:29:55 -08:00
Minu Hong
854d99da52
Update README.md (#1428) 
Update the version number in the dependency for KCL 2.x
 2025-01-03 15:11:04 -08:00
dependabot[bot]
d9a238d33d
Bump com.fasterxml.jackson.core:jackson-databind (#1400) 
Bumps [com.fasterxml.jackson.core:jackson-databind](https://github.com/FasterXML/jackson) from 2.10.1 to 2.12.7.1.
- [Commits](https://github.com/FasterXML/jackson/commits)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.core:jackson-databind
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-12-30 12:25:55 -08:00
vincentvilo-aws
dcac19e927
update dependabot.yml to track master and v2.x branches (#1416) 2024-12-20 10:02:54 -08:00
lucienlu-aws
9da91cd43c
Update version notice to just recommend to use latest version (#1415) 2024-12-18 11:33:06 -08:00
chenylee-aws
4492e1e206
Update version to 3.0.2-SNAPSHOT (#1404) 2024-11-15 12:02:50 -08:00
chenylee-aws
01d0e5cc55
Preparation for v3.0.1 (#1402) 2024-11-14 16:01:36 -08:00
chenylee-aws
3facf303bf
Fix checkpointOwner copy issue for multistream lease (#1401) 2024-11-14 13:50:22 -08:00
Aravinda Kidambi Srinivasan
b154acf7f5
Address KCLv3 issues from github (#1398) 
* Address KCLv3 issues reported on github
1. Fix transitive dependencies and add a maven plugin to catch these
   at build time
2. Remove the redundant shutdown of the leaseCoordinatorThreadPool
3. Fix typo THROUGHOUT_PUT_KBPS
4. Fix shutdown sequence - make sure
    scheduler shutdown without invoking run works
5. Fix backward compatibility check - Avoid flagging methods as deleted
    if it is marked synchronized. Also mark interfaces introduced in KCLv3 as internal.
 2024-11-12 12:20:16 -08:00
Aravinda Kidambi Srinivasan
004f6d3ea0
Update version to 3.0.1-SNAPSHOT (#1394) 
Change version to 3.0.1-SNAPSHOT
 2024-11-07 14:01:51 -08:00
Aravinda Kidambi Srinivasan
f7e909c979
Update backward compatibility check script (#1393) 
Dont fail the check for major release but print the output
for information/review purposes to know what API changes
have occurred in a major release

Manually tested the changes with some incompatible API changes
between patch release, minor release and major release.
 2024-11-06 14:31:25 -08:00
lucienlu-aws
fbfd74df39
Release v3.0.0 (#1392) 2024-11-05 22:09:59 -08:00
dependabot[bot]
a159fa31fb
Bump com.google.protobuf:protobuf-java in /amazon-kinesis-client (#1383) 
Bumps [com.google.protobuf:protobuf-java](https://github.com/protocolbuffers/protobuf) from 4.27.0 to 4.27.5.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-09-30 13:30:44 -07:00
lucienlu-aws
0478575b3d
bugfix: debug or trace logging level now properly logs all worker loops (#1372) 2024-07-26 10:03:46 -07:00
lucienlu-aws
1c0c41c4e8
Add config to enable PITR (#1365) 
* Add config to enable PITR
 2024-07-15 14:15:52 -07:00
nakulj
5878ba8ac6
Add a backwards compatibility check for .proto files (#1362) 2024-07-12 11:18:54 -07:00
nakulj
3b5c59ec04
fix backwards compatibility check (#1361) 2024-07-12 09:46:49 -07:00
nakulj
57dcddf10b
Fix some of the warnings emitted by maven during build (#1363) 
* Set the source encoding in the pom.

This stops the following warning from being emitted in our logs:

```
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
```

* Replace deprecated systemProperties.

The recommended replacement is systemPropertyVariables, documented [here](https://maven.apache.org/surefire/maven-surefire-plugin/examples/system-properties.html).
 2024-07-08 15:22:24 -07:00
Nakul Joshi
f507066ea6 *Actually* skip tests while running backwards compatibility test. 2024-07-02 14:00:57 -07:00
Aravinda Kidambi Srinivasan
6cba7f431d
Remove ShutdownNotificationAware and update javadocs (#1358) 
* Deprecate ShutdownNotificationAware and update javadocs

ShutdownNotificationAware is not used by KCL, this PR
marks it as deprecated and updates the javadoc

Co-authored-by: nakulj <nklj@amazon.com>
 2024-07-02 13:28:19 -07:00
lucienlu-aws
715690d2c0
bugfix: revert update of in memory lease if DDB lease renewal throws error (#1354) 2024-06-24 10:47:00 -07:00
dependabot[bot]
78fb42ede1 Bump commons-io:commons-io from 2.15.1 to 2.16.1 
Bumps commons-io:commons-io from 2.15.1 to 2.16.1.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-07 11:06:32 -07:00
dependabot[bot]
b271fed18b Bump awssdk.version from 2.25.11 to 2.25.64 
Bumps `awssdk.version` from 2.25.11 to 2.25.64.

Updates `software.amazon.awssdk:kinesis` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:dynamodb` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:cloudwatch` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:netty-nio-client` from 2.25.11 to 2.25.64

Updates `software.amazon.awssdk:sts` from 2.25.11 to 2.25.64

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-07 11:05:55 -07:00
dependabot[bot]
cbed8c3c6f Bump com.google.protobuf:protobuf-java from 3.21.12 to 4.27.0 
Bumps [com.google.protobuf:protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.12 to 4.27.0.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-04 10:04:19 -07:00
nakulj
e8c3c01b12
Generate wrappers from proto files instead of shipping them directly (#1340) 
* Replace generated code with source .proto files

This change replaces the generated Messages.java file with the source .proto files.
It also includes the pom.xml changes needed to install protoc from Maven Central, and use it to compile the wrapper java code.

* add explanatory comment
 2024-06-03 15:20:07 -07:00
dependabot[bot]
a83b3fbd57 Bump org.apache.maven.plugins:maven-javadoc-plugin from 3.6.3 to 3.7.0 
Bumps [org.apache.maven.plugins:maven-javadoc-plugin](https://github.com/apache/maven-javadoc-plugin) from 3.6.3 to 3.7.0.
- [Release notes](https://github.com/apache/maven-javadoc-plugin/releases)
- [Commits](https://github.com/apache/maven-javadoc-plugin/compare/maven-javadoc-plugin-3.6.3...maven-javadoc-plugin-3.7.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-javadoc-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-03 09:06:16 -07:00
dependabot[bot]
0005670513 Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.5 
Bumps [org.apache.maven.plugins:maven-failsafe-plugin](https://github.com/apache/maven-surefire) from 3.1.2 to 3.2.5.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.1.2...surefire-3.2.5)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-failsafe-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-03 09:05:50 -07:00
dependabot[bot]
35b150009a Bump org.slf4j:slf4j-api from 2.0.7 to 2.0.13 
Bumps org.slf4j:slf4j-api from 2.0.7 to 2.0.13.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-31 13:18:54 -07:00
dependabot[bot]
d3b84dc995 Bump io.reactivex.rxjava3:rxjava from 3.1.6 to 3.1.8 
Bumps [io.reactivex.rxjava3:rxjava](https://github.com/ReactiveX/RxJava) from 3.1.6 to 3.1.8.
- [Release notes](https://github.com/ReactiveX/RxJava/releases)
- [Commits](https://github.com/ReactiveX/RxJava/compare/v3.1.6...v3.1.8)

---
updated-dependencies:
- dependency-name: io.reactivex.rxjava3:rxjava
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-31 12:56:25 -07:00
Nakul Joshi
49fa32d68e Update maven compiler plugin to 3.13.0 
This version of the compiler plugin supports the `--release` option as
documented [here](https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-release.html).
 2024-05-30 09:33:41 -07:00
Nakul Joshi
221db0564d Fix checkstyle violations. 
Follow up to #1332.
- Manually split up a long string literal.
- Disable ArrayTrailingComma, which checkstyle complains about even in
  commented code. In any case it is redundant with PJF.
 2024-05-21 16:35:31 -07:00
Nakul Joshi
adcc89f80d ignore format commit in blame 2024-05-21 01:27:20 -07:00
Nakul Joshi
18fe49eed0 run spotless:apply 2024-05-21 01:27:20 -07:00
Nakul Joshi
409a8f13cd add spotless 2024-05-21 01:27:20 -07:00
Nakul Joshi
c5e4fa2de0 remove commented imports 2024-05-21 01:27:20 -07:00
dependabot[bot]
a8abbea29f Bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.5 
Bumps [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) from 3.1.2 to 3.2.5.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.1.2...surefire-3.2.5)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-20 16:21:00 -07:00
dependabot[bot]
2648123a79 Bump org.apache.maven.plugins:maven-checkstyle-plugin 
Bumps [org.apache.maven.plugins:maven-checkstyle-plugin](https://github.com/apache/maven-checkstyle-plugin) from 3.3.0 to 3.3.1.
- [Commits](https://github.com/apache/maven-checkstyle-plugin/compare/maven-checkstyle-plugin-3.3.0...maven-checkstyle-plugin-3.3.1)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-checkstyle-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-05-20 16:19:38 -07:00
vincentvilo-aws
023f6be251
Modify backwards compatibility script to ignore synthetic access methods (#1330) 
* modify backwards compatibility script to ignore synthetic access methods

* ignore synthetic access methods starting at access\$000
 2024-05-20 13:06:24 -07:00
nakulj
71f31e0a11
Remove unnecessary lambda (#1329) 
> Returning a lambda from a helper method or saving it in a constant is unnecessary; prefer to implement the functional interface method directly and use a method reference instead.

https://errorprone.info/bugpattern/UnnecessaryLambda
 2024-05-09 15:22:45 -07:00
Aravinda Kidambi Srinivasan
16e8404dc4
Fix a race condition between ShardConsumer shutdown and initialization (#1319) 
* Fix a race condition between ShardConsumer shutdown and initialization

When Kinesis shards have no data, there can be a race condition where
the shard-end record processing from RecordProcessorThread
interleaves with Scheduler performing initialization.
This leads to ShardConsumer making incorrect state transition
during initialization (moves from PROCESSING -> SHUTTING_DOWN) state
and during shutdown handling it moves from SHUTTING_DOWN -> SHUTDOWN_COMPLETE
without running the ShutdownTask.

This can cause the ShardConsumer to not perform proper shutdown
processing that is required for a child shard processing
to be unblocked. So the child shard could be blocked forever unless the
lease for the parent shard moves to a new worker and that worker does
not run into the race condition.

This patch fixes the race condition as follows:

The intializationComplete invocation is not needed after
needsInitialization has been set to false. Because initializationComplete
is mean to perform initialization in an async manner, but once
its done, the async task is a no-op in happy-path, but it can
perform incorrect state transition during a race condition.
 2024-05-02 14:54:59 -07:00
chenylee-aws
69cf5996c5
Honor lease sync on app bootstrap (#1325) 2024-05-02 14:43:03 -07:00
kcl-release-automation-bot
35fc72b2c8
Add snapshot to version 2.6.1-SNAPSHOT (#1324) 2024-05-01 17:54:42 -07:00
kcl-release-automation-bot
89a90e34e2
Preparation for v2.6.0 (#1323) 2024-05-01 17:48:25 -07:00
lucienlu-aws
ab24b66039
Skip cross account tests if no cross account credentials are provided (#1321) 2024-04-30 16:56:05 -07:00
Brendan Lynch
34fe58c492
Get unassigned leases in leasesToTake (#1320) 
Consider all null leases as a possible lease to take alongside expired leases

---------

Co-authored-by: Brendan Lynch <brenplyn@amazon.com>
 2024-04-30 15:56:31 -07:00
furq-aws
ec34ed1def
Internally construct and use stream ARNs for all streams in multi-stream mode (#1318) 2024-04-30 14:19:58 -07:00
lucienlu-aws
c12cee2a1b
Add additional integration tests for multistream and cross account (#1313) 2024-04-30 11:16:37 -07:00
lucienlu-aws
96be30b3e7
Change agedFailoverTimeMultiplier config to doPriorityLeaseTaking (#1317) 
* Change agedFailoverTimeMultiplier config to doPriorityLeaseTaking
 2024-04-22 10:35:08 -07:00
vincentvilo-aws
7d55b7a77f
Create script to check for backwards compatibility (#1316) 
* create script to check for backwards compatibility

* move scripts directory to .github folder

* add execute permissions to script

* moved the 'continue' call to the find_removed_methods() function

* add function to check if minor release is being performed

* Upgrade version to 2.6.0-SNAPSHOT
 2024-04-19 14:01:04 -07:00
furq-aws
e9990190cc
Update RetrievalFactory implementations to utilize the StreamIdentifier field of StreamConfig (#1291) 2024-04-15 11:14:13 -07:00