luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
487 commits 48 branches 124 tags 11 MiB
Commit graph

487 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ryan Pelaez
a3d10bf702 Updated arn import to use software.amzon instead of com.amazonaws, also updated unit tests to be more explicit with the expected exceptions 2023-06-14 17:26:27 -07:00
Ryan Pelaez
d2fe79112e Updated multilang to support streamArn 2023-06-14 16:46:50 -07:00
furq-aws
a8fc1367c6
Update KCL version to 2.5.1-SNAPSHOT (#1115) 2023-05-19 19:45:31 -07:00
furq-aws
a35c3a1599
2.5.0 Release Prep: Add quotations to RetrievalConfig KCLversion and remove CHANGELOG whitespace 2023-05-19 17:12:20 -07:00
lucienlu-aws
8b3f957db4
Preparation for v2.5.0 (#1113) 
* Preparation for v2.5.0
 2023-05-19 16:25:45 -07:00
furq-aws
d7f3a079e1
Add support for stream ARNs (#1109) 
Add support for referencing streams by streamARN in single-stream mode, or by the combination of streamARN and creationEpoch in multi-stream mode.
 2023-05-19 12:21:20 -07:00
dependabot[bot]
7092ffdbd6
Bump nexus-staging-maven-plugin from 1.6.8 to 1.6.13 (#1072) 
Bumps nexus-staging-maven-plugin from 1.6.8 to 1.6.13.

---
updated-dependencies:
- dependency-name: org.sonatype.plugins:nexus-staging-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 15:32:01 -07:00
dependabot[bot]
32a29dbf49
Bump slf4j-api from 2.0.6 to 2.0.7 (#1073) 
Bumps [slf4j-api](https://github.com/qos-ch/slf4j) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/commits)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:36:00 -07:00
dependabot[bot]
58ceaf4526
Bump awssdk.version from 2.20.40 to 2.20.43 (#1090) 
Bumps `awssdk.version` from 2.20.40 to 2.20.43.

Updates `kinesis` from 2.20.40 to 2.20.43

Updates `dynamodb` from 2.20.40 to 2.20.43

Updates `cloudwatch` from 2.20.40 to 2.20.43

Updates `netty-nio-client` from 2.20.40 to 2.20.43

Updates `sts` from 2.20.40 to 2.20.43

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-16 14:29:20 -07:00
dependabot[bot]
53c76c72c3
Bump maven-compiler-plugin from 3.8.1 to 3.11.0 (#1071) 
Bumps [maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) from 3.8.1 to 3.11.0.
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.8.1...maven-compiler-plugin-3.11.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-15 14:16:37 -07:00
stair
ee3f56ae66
Added missing copyright to new files. (#1106) 2023-05-05 14:29:34 -04:00
stair
f115235fd1
Refactored SynchronizedCache out of SupplierCache, and introduced (#1107) 
`FunctionCache`.
 2023-05-05 11:19:27 -07:00
stair
e3d845a1f5
StreamARN: removed region from StreamIdentifier serialization. (#1099) 
Provided ARNs must share the same region as the Kinesis endpoint.
 2023-04-20 19:18:14 -04:00
stair
b86fa22e96
StreamARN: enhanced StreamIdentifier to spawn instances from stream… (#1098) 
* StreamARN: enhanced `StreamIdentifier` to spawn instances from stream ARNs.

* Bugfix: moved instantation of `Future` inside `Supplier`.
 2023-04-19 19:38:00 -04:00
stair
0fd94acb2b
StreamARN: fast-follow to invent-and-simplify https://github.com/awslabs/amazon-kinesis-client/pull/1087 (#1097) 
* fixed memory leak in `StreamARNUtil` (new class)
* substantial DRY
* added more, and enhanced recently-provided, unit tests
 2023-04-18 16:29:24 -04:00
Yu Zeng
52e34dbe8f
Internally construct StreamARN using STS (#1087) 
Co-authored-by: Yu Zeng <yuzen@amazon.com>
Co-authored-by: stair <123031771+stair-aws@users.noreply.github.com>
 2023-04-18 15:26:31 -04:00
stair
fc52976c3d
Added SupplierCache (so it may be leveraged by StreamARN work). (#1096) 2023-04-18 14:58:46 -04:00
stair
5e7d4788ec
Code cleanup to introduce better testing and simplify future removal of (#1094) 
deprecated parameters (e.g., `Either<L, R> appStreamTracker`).
 2023-04-18 14:58:27 -04:00
chenylee-aws
7b23ae9b3c
Minimize race in PSSM to avoid unnecessary shard sync calls (#1088) 2023-04-12 10:33:50 -07:00
Yu Zeng
7cd7c27a80
Bump awssdk.version from 2.20.8 to 2.20.40 (#1089) 
Co-authored-by: Yu Zeng <yuzen@amazon.com>
 2023-04-10 16:27:11 -07:00
Eric Meisel
88246e717e
Add simple SingleStreamTracker constructor with position (#1086) 2023-03-26 09:24:22 -04:00
furq-aws
b8d3390bf3
Fix flaky restartAfterRequestTimerExpires tests (#1084) 
Add wait to allow subscriptions to start.
This eliminates flakiness of tests restartAfterRequestTimerExpiresWhenNotGettingRecordsAfterInitialization() and restartAfterRequestTimerExpiresWhenInitialTaskExecutionIsRejected().
 2023-03-24 04:53:44 -07:00
noahbt
10cdf43b9d
Updated versions to SNAPSHOT (#1085) 2023-03-23 13:54:50 -04:00
Ryan French
2ecd1c4ac5
Allow tags to be added when lease table is created (#1065) 
* Allow tags to be added when lease table is created

* Add javadoc comment to new member variable

* DRY up creation of the CreateTable Request builder

* Fix compiler error

* Remove unnecessary eq functions

* Fix indentation

* Add patch
 2023-03-23 13:26:07 -04:00
furq-aws
0627ba50bb
Fix flaky HashRangesAreAlwaysComplete test (#1066) 
Updates depth value for Merge and Reshard with some In-Progress Parents tests to prevent invalid hierarchy trees.
 2023-03-22 09:33:28 -07:00
noahbt
b894669bda
Changes made for 2.4.8 (#1083) 2023-03-21 17:17:12 -07:00
stair
6be92dc4ef
Added metrics in ShutdownTask for scenarios when parent leases are missing. (#1080) 
+ optimizations in `ShutdownTask` (e.g., `Random` static instance,
eliminated over-used Function)
+ DRY+KISS on `ShutdownTaskTest`
+ deleted some dead code
 2023-03-21 19:52:17 -04:00
noahbt
177303d557
Changelog updated for 2.4.7 (#1078) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:21:54 -07:00
noahbt
26a692530d
Revert changes to pom property (#1077) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:03:36 -07:00
furq-aws
9d07af403d
Fix flaky InitializationWaitsWhenLeaseTableIsEmpty test (#1069) 
Update misconfigured Scheduler MIN_WAIT_TIME_FOR_LEASE_TABLE_CHECK_MILLIS test value to be in sync with source value
 2023-03-20 14:00:05 -07:00
noahbt
0d5007c04c
Files updated for KCL release 2.4.7 (#1067) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-17 13:21:13 -07:00
dependabot[bot]
9ff99f0182
Bump awssdk.version from 2.19.31 to 2.20.8 (#1047) 
Bumps `awssdk.version` from 2.19.31 to 2.20.8.

Updates `kinesis` from 2.19.31 to 2.20.8

Updates `dynamodb` from 2.19.31 to 2.20.8

Updates `cloudwatch` from 2.19.31 to 2.20.8

Updates `netty-nio-client` from 2.19.31 to 2.20.8

Updates `sts` from 2.19.31 to 2.20.8

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:18:31 -07:00
dependabot[bot]
bc71990bec
Bump maven-javadoc-plugin from 3.3.1 to 3.5.0 (#1046) 
Bumps [maven-javadoc-plugin](https://github.com/apache/maven-javadoc-plugin) from 3.3.1 to 3.5.0.
- [Release notes](https://github.com/apache/maven-javadoc-plugin/releases)
- [Commits](https://github.com/apache/maven-javadoc-plugin/compare/maven-javadoc-plugin-3.3.1...maven-javadoc-plugin-3.5.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-javadoc-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:14:31 -07:00
dependabot[bot]
7416f8b915
Bump gsr.version from 1.1.13 to 1.1.14 (#1038) 
Bumps `gsr.version` from 1.1.13 to 1.1.14.

Updates `schema-registry-serde` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

Updates `schema-registry-common` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

---
updated-dependencies:
- dependency-name: software.amazon.glue:schema-registry-serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.glue:schema-registry-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:11:05 -07:00
dependabot[bot]
0a3fcb07bd
Bump aws-java-sdk.version from 1.12.370 to 1.12.405 (#1037) 
Bumps `aws-java-sdk.version` from 1.12.370 to 1.12.405.

Updates `aws-java-sdk-core` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

Updates `aws-java-sdk-sts` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

---
updated-dependencies:
- dependency-name: com.amazonaws:aws-java-sdk-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.amazonaws:aws-java-sdk-sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:09:46 -07:00
chenylee-aws
0cbd74f6e7
Allow leader to learn new leases upon re-election to avoid unnecessary shardSyncs (#1063) 2023-03-15 12:34:32 -07:00
noahbt
04a121a811
Add new metric to be emitted on lease creation (#1060) 
* Add new metric to be emitted on lease creation

* Rebase changes from master

---------

Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-14 14:09:47 -07:00
stair
5bbb9768b5
DRY: simplification of HierarchicalShardSyncerTest. (#1059) 2023-03-13 20:05:40 -04:00
stair
c4204002af
Fixed retry storm in PrefetchRecordsPublisher. (#1062) 
+ DRY in `PrefetchRecordsPublisherTest`
 2023-03-13 19:57:01 -04:00
stair
504ea10859
Fixed NPE in LeaseCleanupManager. (#1061) 2023-03-10 15:07:46 -05:00
Abhit Sawwalakhe
27b166c5aa
Clean up in-memory state of deleted kinesis stream in MultiStreamMode (#1056) 
Co-authored-by: Abhit Sawwalakhe <sawwa@amazon.com>
 2023-03-08 13:39:35 -08:00
stair
43d43653d0
Documentation: added <pre> tags so fixed-format diagrams aren't garbled. (#1058) 
No functional change.
 2023-03-07 15:36:25 -05:00
stair
87dc586e6d
Added more logging in Scheduler w.r.t. StreamConfigs. (#1057) 2023-03-06 17:05:34 -05:00
stair
cd80c93966
Exposed convenience method of ExtendedSequenceNumber#isSentinelCheckpoint() (#1053) 
+ fixed unrelated parameterized log message in `ShardSyncTaskManager`
 2023-02-27 10:15:30 -08:00
stair
d8aa784f17
Removed a .swp file, and updated .gitignore. (#1043) 2023-02-27 10:02:36 -08:00
ZeyuLi-AWS
70cfc7d7ce
Fixed duplication of project version in children pom.xml (#1045) 2023-02-20 15:53:57 -08:00
stair
1c8bd8e71e
Minor optimizations (e.g., calculate-once, put instead of get+put) (#1041) 
and code cleanup (e.g., removed unused imports, updated Javadoc).

No functional change.
 2023-02-13 10:58:02 -08:00
stair
5bfd1ab289
Release Note updates to avoid duplication and bitrot (e.g., 1.x release (#1035) 
notes are not ported forward to 2.x).
 2023-02-13 13:20:54 -05:00
stair
4d94efac8f
Optimization: 9~15% improvement in KinesisDataFetcher wall-time after (#1034) 
converting `AWSExceptionManger` to a static variable.
 2023-02-13 13:16:28 -05:00
pelaezryan
9fb58a22bf
Increased logging verbosity around lease management. Also included additional javadocs for methods (#1040) 
Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-02-13 12:28:54 -05:00