luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
500 commits 48 branches 124 tags 11 MiB
Commit graph

500 commits

This branch
This branch
All branches
Author SHA1 Message Date
Meher Mankikar
9460889b3c Updating comments with bug fix, test cases were failing when being run in succession becuase of incorrect closing of threads 2023-08-01 17:17:10 -07:00
Meher Mankikar
dc0fa21b4a Updates based off initial comments 2023-06-29 14:50:09 -07:00
Meher M
a207c3760c
Merge branch 'awslabs:master' into KCL2XReshardTests 2023-06-28 10:51:21 -07:00
stair
feadd5e043
Fix NPE on graceful shutdown before DDB LeaseCoordinator starts. (#1157) 2023-06-28 10:36:32 -04:00
stair
a9b0d00852
Checkstyle: added additional checks to, primarily, safeguard against bugs. (#1154) 2023-06-27 15:24:03 -04:00
pelaezryan
4eff398147
Preparation for v2.5.1 (#1155) 
* Preparation for minor version v2.5.1

---------

Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-06-27 10:21:49 -07:00
stair
768f6a36bb
Checkstyle: added UnusedImports check. (#1153) 2023-06-26 16:19:30 -04:00
stair
74d8f4b780
Enabled Checkstyle validation of test resources. (#1150) 
No functional change.
 2023-06-26 15:25:10 -04:00
dependabot[bot]
5105317eb4
Bump guava from 31.1-jre to 32.0.0-jre in /amazon-kinesis-client (#1142) 
Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-06-26 15:11:34 -04:00
Meher Mankikar
a50c2fcdf8 Initial changes for resharding integration tests KCL 2.x and changing integration tests to not run by default 2023-06-26 11:53:48 -07:00
pelaezryan
dcd1c53fb1
Update to Multilang Daemon to support StreamArn (#1143) 
* Updated multilang to support streamArn

* Updated arn import to use software.amzon instead of com.amazonaws, also updated unit tests to be more explicit with the expected exceptions

* Updated exception wording for region validation in StreamArn to be more consistent with other error messages

* reverted spacing change

* Updated StreamArn in multilang to only replace streamName (not region as well). Also updated unit tests and added Region validation

* Updated region validation in multilang to be more readible

* Refactored multilang unit tests to be more simple

* Updated multilang daemon to validate streamArn based on pattern rather than individual section

* removed region validation as this was not a requirement for stringArn support in multilangdaemon

* removed spacing and removed unit test assertion on exception message

* removed unnecessary param from unit test

* removed unused imports from multilang unit tests

* simplified the assertion for multilang daemon unit tests

* Cleaned up unit test code following best practices for spacing/naming conventions and simplied kinesisClientLibConfiguration

* Updated region code in unit tests for multilang daemon

---------

Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-06-26 09:02:19 -07:00
stair
eb6fd0cf32
Bound Checkstyle to validate goal for automated enforcement. (#1149) 2023-06-23 16:15:33 -04:00
stair
3d6800874c
Code cleanup to faciliate Checkstyle enforcement. (#1148) 
No functional change.
 2023-06-23 14:58:10 -04:00
mmankika-aws
53dbb4ea79
Adding testing architecture and KCL 2.x basic polling/streaming tests (#1136) 
* Adding testing architecture and KCL 2.x basic polling and streaming tests
 2023-06-21 14:55:55 -07:00
mmankika-aws
f1ef0e820d
GitHub actions (#1145) 
* Adding Github Actions config
 2023-06-20 11:12:24 -07:00
furq-aws
a8fc1367c6
Update KCL version to 2.5.1-SNAPSHOT (#1115) 2023-05-19 19:45:31 -07:00
furq-aws
a35c3a1599
2.5.0 Release Prep: Add quotations to RetrievalConfig KCLversion and remove CHANGELOG whitespace 2023-05-19 17:12:20 -07:00
lucienlu-aws
8b3f957db4
Preparation for v2.5.0 (#1113) 
* Preparation for v2.5.0
 2023-05-19 16:25:45 -07:00
furq-aws
d7f3a079e1
Add support for stream ARNs (#1109) 
Add support for referencing streams by streamARN in single-stream mode, or by the combination of streamARN and creationEpoch in multi-stream mode.
 2023-05-19 12:21:20 -07:00
dependabot[bot]
7092ffdbd6
Bump nexus-staging-maven-plugin from 1.6.8 to 1.6.13 (#1072) 
Bumps nexus-staging-maven-plugin from 1.6.8 to 1.6.13.

---
updated-dependencies:
- dependency-name: org.sonatype.plugins:nexus-staging-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 15:32:01 -07:00
dependabot[bot]
32a29dbf49
Bump slf4j-api from 2.0.6 to 2.0.7 (#1073) 
Bumps [slf4j-api](https://github.com/qos-ch/slf4j) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/commits)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:36:00 -07:00
dependabot[bot]
58ceaf4526
Bump awssdk.version from 2.20.40 to 2.20.43 (#1090) 
Bumps `awssdk.version` from 2.20.40 to 2.20.43.

Updates `kinesis` from 2.20.40 to 2.20.43

Updates `dynamodb` from 2.20.40 to 2.20.43

Updates `cloudwatch` from 2.20.40 to 2.20.43

Updates `netty-nio-client` from 2.20.40 to 2.20.43

Updates `sts` from 2.20.40 to 2.20.43

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:29:20 -07:00
dependabot[bot]
53c76c72c3
Bump maven-compiler-plugin from 3.8.1 to 3.11.0 (#1071) 
Bumps [maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) from 3.8.1 to 3.11.0.
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.8.1...maven-compiler-plugin-3.11.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-15 14:16:37 -07:00
stair
ee3f56ae66
Added missing copyright to new files. (#1106) 2023-05-05 14:29:34 -04:00
stair
f115235fd1
Refactored SynchronizedCache out of SupplierCache, and introduced (#1107) 
`FunctionCache`.
 2023-05-05 11:19:27 -07:00
stair
e3d845a1f5
StreamARN: removed region from StreamIdentifier serialization. (#1099) 
Provided ARNs must share the same region as the Kinesis endpoint.
 2023-04-20 19:18:14 -04:00
stair
b86fa22e96
StreamARN: enhanced StreamIdentifier to spawn instances from stream… (#1098) 
* StreamARN: enhanced `StreamIdentifier` to spawn instances from stream ARNs.

* Bugfix: moved instantation of `Future` inside `Supplier`.
 2023-04-19 19:38:00 -04:00
stair
0fd94acb2b
StreamARN: fast-follow to invent-and-simplify https://github.com/awslabs/amazon-kinesis-client/pull/1087 (#1097) 
* fixed memory leak in `StreamARNUtil` (new class)
* substantial DRY
* added more, and enhanced recently-provided, unit tests
 2023-04-18 16:29:24 -04:00
Yu Zeng
52e34dbe8f
Internally construct StreamARN using STS (#1087) 
Co-authored-by: Yu Zeng <yuzen@amazon.com>
Co-authored-by: stair <123031771+stair-aws@users.noreply.github.com>
 2023-04-18 15:26:31 -04:00
stair
fc52976c3d
Added SupplierCache (so it may be leveraged by StreamARN work). (#1096) 2023-04-18 14:58:46 -04:00
stair
5e7d4788ec
Code cleanup to introduce better testing and simplify future removal of (#1094) 
deprecated parameters (e.g., `Either<L, R> appStreamTracker`).
 2023-04-18 14:58:27 -04:00
chenylee-aws
7b23ae9b3c
Minimize race in PSSM to avoid unnecessary shard sync calls (#1088) 2023-04-12 10:33:50 -07:00
Yu Zeng
7cd7c27a80
Bump awssdk.version from 2.20.8 to 2.20.40 (#1089) 
Co-authored-by: Yu Zeng <yuzen@amazon.com>
 2023-04-10 16:27:11 -07:00
Eric Meisel
88246e717e
Add simple SingleStreamTracker constructor with position (#1086) 2023-03-26 09:24:22 -04:00
furq-aws
b8d3390bf3
Fix flaky restartAfterRequestTimerExpires tests (#1084) 
Add wait to allow subscriptions to start.
This eliminates flakiness of tests restartAfterRequestTimerExpiresWhenNotGettingRecordsAfterInitialization() and restartAfterRequestTimerExpiresWhenInitialTaskExecutionIsRejected().
 2023-03-24 04:53:44 -07:00
noahbt
10cdf43b9d
Updated versions to SNAPSHOT (#1085) 2023-03-23 13:54:50 -04:00
Ryan French
2ecd1c4ac5
Allow tags to be added when lease table is created (#1065) 
* Allow tags to be added when lease table is created

* Add javadoc comment to new member variable

* DRY up creation of the CreateTable Request builder

* Fix compiler error

* Remove unnecessary eq functions

* Fix indentation

* Add patch
 2023-03-23 13:26:07 -04:00
furq-aws
0627ba50bb
Fix flaky HashRangesAreAlwaysComplete test (#1066) 
Updates depth value for Merge and Reshard with some In-Progress Parents tests to prevent invalid hierarchy trees.
 2023-03-22 09:33:28 -07:00
noahbt
b894669bda
Changes made for 2.4.8 (#1083) 2023-03-21 17:17:12 -07:00
stair
6be92dc4ef
Added metrics in ShutdownTask for scenarios when parent leases are missing. (#1080) 
+ optimizations in `ShutdownTask` (e.g., `Random` static instance,
eliminated over-used Function)
+ DRY+KISS on `ShutdownTaskTest`
+ deleted some dead code
 2023-03-21 19:52:17 -04:00
noahbt
177303d557
Changelog updated for 2.4.7 (#1078) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:21:54 -07:00
noahbt
26a692530d
Revert changes to pom property (#1077) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:03:36 -07:00
furq-aws
9d07af403d
Fix flaky InitializationWaitsWhenLeaseTableIsEmpty test (#1069) 
Update misconfigured Scheduler MIN_WAIT_TIME_FOR_LEASE_TABLE_CHECK_MILLIS test value to be in sync with source value
 2023-03-20 14:00:05 -07:00
noahbt
0d5007c04c
Files updated for KCL release 2.4.7 (#1067) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-17 13:21:13 -07:00
dependabot[bot]
9ff99f0182
Bump awssdk.version from 2.19.31 to 2.20.8 (#1047) 
Bumps `awssdk.version` from 2.19.31 to 2.20.8.

Updates `kinesis` from 2.19.31 to 2.20.8

Updates `dynamodb` from 2.19.31 to 2.20.8

Updates `cloudwatch` from 2.19.31 to 2.20.8

Updates `netty-nio-client` from 2.19.31 to 2.20.8

Updates `sts` from 2.19.31 to 2.20.8

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:18:31 -07:00
dependabot[bot]
bc71990bec
Bump maven-javadoc-plugin from 3.3.1 to 3.5.0 (#1046) 
Bumps [maven-javadoc-plugin](https://github.com/apache/maven-javadoc-plugin) from 3.3.1 to 3.5.0.
- [Release notes](https://github.com/apache/maven-javadoc-plugin/releases)
- [Commits](https://github.com/apache/maven-javadoc-plugin/compare/maven-javadoc-plugin-3.3.1...maven-javadoc-plugin-3.5.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-javadoc-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:14:31 -07:00
dependabot[bot]
7416f8b915
Bump gsr.version from 1.1.13 to 1.1.14 (#1038) 
Bumps `gsr.version` from 1.1.13 to 1.1.14.

Updates `schema-registry-serde` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

Updates `schema-registry-common` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

---
updated-dependencies:
- dependency-name: software.amazon.glue:schema-registry-serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.glue:schema-registry-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:11:05 -07:00
dependabot[bot]
0a3fcb07bd
Bump aws-java-sdk.version from 1.12.370 to 1.12.405 (#1037) 
Bumps `aws-java-sdk.version` from 1.12.370 to 1.12.405.

Updates `aws-java-sdk-core` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

Updates `aws-java-sdk-sts` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

---
updated-dependencies:
- dependency-name: com.amazonaws:aws-java-sdk-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.amazonaws:aws-java-sdk-sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:09:46 -07:00
chenylee-aws
0cbd74f6e7
Allow leader to learn new leases upon re-election to avoid unnecessary shardSyncs (#1063) 2023-03-15 12:34:32 -07:00
noahbt
04a121a811
Add new metric to be emitted on lease creation (#1060) 
* Add new metric to be emitted on lease creation

* Rebase changes from master

---------

Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-14 14:09:47 -07:00