luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
519 commits 48 branches 124 tags 11 MiB
Commit graph

519 commits

This branch
This branch
All branches
Author SHA1 Message Date
Aravinda Kidambi Srinivasan
d39bde3b71 Fix an issue in configuring idleTimeBetweenReadsInMillis in MultiLangDaemon 
Fix an issue where the idleTimeBetweenReadInMillis configured
via MultiLangDaemon was not taking effect because it used
the auto-generated setter from Lombok to set the configured value,
while there is a custom setter that must be invoked to set the
value correctly.

There is also a general confusion between using Lombok's setter vs
custom setter in java.

Unifying the approach to use the custom Lombok-fluent-style setter
and deprecating the previously added custom setIdleTimeBetweenReadsInMillis

Correct way to configure idleTimeBetweenReadsInMillis for MultiLang is
to add this in the properties file:
idleTimeBetweenReadsInMillis = 10000 # 10 seconds

Correct way to configure for java:
configsBuilder.retrievalConfig().retrievalSpecificConfig(
    new PollingConfig(streamName, kinesisClient)
        .idleTimeBetweenReadsInMillis(Duration.ofSeconds(10).toMillis())

Issues: #999, #950, #515
 2023-11-22 12:43:07 -08:00
stair
a48f5436ee
Added link to javadoc.io-hosted Javadoc. (#1229) 2023-11-16 17:37:50 -05:00
stair
51a62a559c
Added doc for leases and the lease lifecycle. (#1218) 
* Added doc for leases and the lease lifecycle.

* Documentation: addressed comments for leases.
+ minor code cleanup
* Documentation: language review.
+ decomposed shard sync UML into two separate diagrams (initialization, loop)
* Documentation: language review touch-ups.
 2023-11-10 14:56:15 -05:00
Brendan Lynch
8ed4999a46
Revbump KCL from 2.5.3 to 2.5.4-SNAPSHOT (#1226) 
---------

Co-authored-by: Brendan Lynch <brenplyn@amazon.com>
 2023-11-09 10:42:39 -08:00
Brendan Lynch
a7694a1f67
Release of 2.5.3 (#1223) 2023-11-08 16:14:15 -08:00
furq-aws
f90b1b1c05
Provide streamArn in getRecords request (#1219) 2023-10-26 16:49:07 -07:00
Matt Dziuban
118783b18b
Update gsr.version to 1.1.17 (#1216) 2023-10-25 15:27:51 -07:00
Brendan Lynch
cf5e1e4c7f
Bumped aws-glue-schema-registry version (#1215) 
Co-authored-by: Brendan Lynch <brenplyn@amazon.com>
 2023-10-11 14:48:42 -07:00
stair
7899820cb1
FAQ: Augmented "What is the impact ..." (#1205) 2023-09-07 13:46:35 -04:00
stair
0aff42f8fe
Added links from README.md to FAQ and doc folder. (#1203) 2023-08-30 13:18:57 -04:00
stair
aabcbaf4b7
Initial FAQ, with hopefully more contributions in the future. (#1202) 2023-08-30 13:07:23 -04:00
vincentvilo-aws
78b565fa9b
add test case for StreamIdentifier serialization (#1200) 2023-08-22 14:34:12 -07:00
stair
a1731dc49b
Reintroduced -SNAPSHOT classifier on KCL version. (#1188) 2023-08-07 18:13:07 -07:00
stair
7384bc1dbe
Release of 2.5.2 (#1187) 2023-08-07 17:54:53 -07:00
stair
12b9a36d0d
Provided documentation for multilang's new NestedPropertyKey enhancement. (#1186) 2023-08-07 16:56:25 -04:00
stair
2f4ff65681
[#367] Enhanced multi-lang AWSCredentialsProvider=... decoder and c… (#1184) 
* [#367] Enhanced multi-lang `AWSCredentialsProvider=...` decoder and construction.

+ added support for external ids (issue #367)
+ added support for endpoint+region (e.g., STS via VPC)

* Multiple multi-lang edits to introduce logging and additional tests.

+ added `ENDPOINT_REGION` nested key for a simpler Cx experience
+ deduplicated, and improved, logic w.r.t. CredentialsProvider
construction to NOT swallow Exceptions

* Relocated `multilang.properties` from `main/resources` to `test/resources`
 2023-08-07 16:29:49 -04:00
Meher M
46cd1179d4
Adding resharding integration tests and changing ITs to not run by default (#1152) 
* Initial changes for resharding integration tests KCL 2.x and changing integration tests to not run by default
 2023-08-03 13:16:56 -07:00
stair
eccd6cf2e7
CVE-2023-2976: revbump Guava 32.0.0-jre -> 32.1.1-jre (#1181) 
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-2976
 2023-07-17 13:08:27 -07:00
pelaezryan
b2c3f9712a
Bumped KCL Version from 2.5.1 to 2.5.2-SNAPSHOT (#1164) 
* Bumped KCL Version from 2.5.1 to 2.5.2-SNAPSHOT

* Updated pom.xml for amazon-kinesis-client and amazon-kinesis-client-multilang to 2.5.2-SNAPSHOT

---------

Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-07-07 15:35:08 -04:00
Meher M
c3883f5763
Only deleting resource created by ITs (#1162) 2023-07-06 16:47:15 -07:00
stair
8d1ee6b5e1
Checkstyle: tightened LineLength restriction from 170 to 150. (#1158) 2023-07-06 10:24:14 -07:00
Meher M
42eb753d62
Bug fix in lease refresher integration test with occasional failures (#1159) 2023-06-29 21:51:07 -07:00
stair
290facdd51
Modified dependabot.yml to set the correct v[1|2].x label. (#1151) 2023-06-28 16:13:15 -04:00
stair
feadd5e043
Fix NPE on graceful shutdown before DDB LeaseCoordinator starts. (#1157) 2023-06-28 10:36:32 -04:00
stair
a9b0d00852
Checkstyle: added additional checks to, primarily, safeguard against bugs. (#1154) 2023-06-27 15:24:03 -04:00
pelaezryan
4eff398147
Preparation for v2.5.1 (#1155) 
* Preparation for minor version v2.5.1

---------

Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-06-27 10:21:49 -07:00
stair
768f6a36bb
Checkstyle: added UnusedImports check. (#1153) 2023-06-26 16:19:30 -04:00
stair
74d8f4b780
Enabled Checkstyle validation of test resources. (#1150) 
No functional change.
 2023-06-26 15:25:10 -04:00
dependabot[bot]
5105317eb4
Bump guava from 31.1-jre to 32.0.0-jre in /amazon-kinesis-client (#1142) 
Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-26 15:11:34 -04:00
pelaezryan
dcd1c53fb1
Update to Multilang Daemon to support StreamArn (#1143) 
* Updated multilang to support streamArn

* Updated arn import to use software.amzon instead of com.amazonaws, also updated unit tests to be more explicit with the expected exceptions

* Updated exception wording for region validation in StreamArn to be more consistent with other error messages

* reverted spacing change

* Updated StreamArn in multilang to only replace streamName (not region as well). Also updated unit tests and added Region validation

* Updated region validation in multilang to be more readible

* Refactored multilang unit tests to be more simple

* Updated multilang daemon to validate streamArn based on pattern rather than individual section

* removed region validation as this was not a requirement for stringArn support in multilangdaemon

* removed spacing and removed unit test assertion on exception message

* removed unnecessary param from unit test

* removed unused imports from multilang unit tests

* simplified the assertion for multilang daemon unit tests

* Cleaned up unit test code following best practices for spacing/naming conventions and simplied kinesisClientLibConfiguration

* Updated region code in unit tests for multilang daemon

---------

Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-06-26 09:02:19 -07:00
stair
eb6fd0cf32
Bound Checkstyle to validate goal for automated enforcement. (#1149) 2023-06-23 16:15:33 -04:00
stair
3d6800874c
Code cleanup to faciliate Checkstyle enforcement. (#1148) 
No functional change.
 2023-06-23 14:58:10 -04:00
mmankika-aws
53dbb4ea79
Adding testing architecture and KCL 2.x basic polling/streaming tests (#1136) 
* Adding testing architecture and KCL 2.x basic polling and streaming tests
 2023-06-21 14:55:55 -07:00
mmankika-aws
f1ef0e820d
GitHub actions (#1145) 
* Adding Github Actions config
 2023-06-20 11:12:24 -07:00
furq-aws
a8fc1367c6
Update KCL version to 2.5.1-SNAPSHOT (#1115) 2023-05-19 19:45:31 -07:00
furq-aws
a35c3a1599
2.5.0 Release Prep: Add quotations to RetrievalConfig KCLversion and remove CHANGELOG whitespace 2023-05-19 17:12:20 -07:00
lucienlu-aws
8b3f957db4
Preparation for v2.5.0 (#1113) 
* Preparation for v2.5.0
 2023-05-19 16:25:45 -07:00
furq-aws
d7f3a079e1
Add support for stream ARNs (#1109) 
Add support for referencing streams by streamARN in single-stream mode, or by the combination of streamARN and creationEpoch in multi-stream mode.
 2023-05-19 12:21:20 -07:00
dependabot[bot]
7092ffdbd6
Bump nexus-staging-maven-plugin from 1.6.8 to 1.6.13 (#1072) 
Bumps nexus-staging-maven-plugin from 1.6.8 to 1.6.13.

---
updated-dependencies:
- dependency-name: org.sonatype.plugins:nexus-staging-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 15:32:01 -07:00
dependabot[bot]
32a29dbf49
Bump slf4j-api from 2.0.6 to 2.0.7 (#1073) 
Bumps [slf4j-api](https://github.com/qos-ch/slf4j) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/commits)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:36:00 -07:00
dependabot[bot]
58ceaf4526
Bump awssdk.version from 2.20.40 to 2.20.43 (#1090) 
Bumps `awssdk.version` from 2.20.40 to 2.20.43.

Updates `kinesis` from 2.20.40 to 2.20.43

Updates `dynamodb` from 2.20.40 to 2.20.43

Updates `cloudwatch` from 2.20.40 to 2.20.43

Updates `netty-nio-client` from 2.20.40 to 2.20.43

Updates `sts` from 2.20.40 to 2.20.43

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:29:20 -07:00
dependabot[bot]
53c76c72c3
Bump maven-compiler-plugin from 3.8.1 to 3.11.0 (#1071) 
Bumps [maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) from 3.8.1 to 3.11.0.
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.8.1...maven-compiler-plugin-3.11.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-15 14:16:37 -07:00
stair
ee3f56ae66
Added missing copyright to new files. (#1106) 2023-05-05 14:29:34 -04:00
stair
f115235fd1
Refactored SynchronizedCache out of SupplierCache, and introduced (#1107) 
`FunctionCache`.
 2023-05-05 11:19:27 -07:00
stair
e3d845a1f5
StreamARN: removed region from StreamIdentifier serialization. (#1099) 
Provided ARNs must share the same region as the Kinesis endpoint.
 2023-04-20 19:18:14 -04:00
stair
b86fa22e96
StreamARN: enhanced StreamIdentifier to spawn instances from stream… (#1098) 
* StreamARN: enhanced `StreamIdentifier` to spawn instances from stream ARNs.

* Bugfix: moved instantation of `Future` inside `Supplier`.
 2023-04-19 19:38:00 -04:00
stair
0fd94acb2b
StreamARN: fast-follow to invent-and-simplify https://github.com/awslabs/amazon-kinesis-client/pull/1087 (#1097) 
* fixed memory leak in `StreamARNUtil` (new class)
* substantial DRY
* added more, and enhanced recently-provided, unit tests
 2023-04-18 16:29:24 -04:00
Yu Zeng
52e34dbe8f
Internally construct StreamARN using STS (#1087) 
Co-authored-by: Yu Zeng <yuzen@amazon.com>
Co-authored-by: stair <123031771+stair-aws@users.noreply.github.com>
 2023-04-18 15:26:31 -04:00
stair
fc52976c3d
Added SupplierCache (so it may be leveraged by StreamARN work). (#1096) 2023-04-18 14:58:46 -04:00
stair
5e7d4788ec
Code cleanup to introduce better testing and simplify future removal of (#1094) 
deprecated parameters (e.g., `Either<L, R> appStreamTracker`).
 2023-04-18 14:58:27 -04:00