luciano/amazon-kinesis-client
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
616 commits 48 branches 124 tags 11 MiB
Commit graph

616 commits

This branch
This branch
All branches
Author SHA1 Message Date
furq-aws
b8d3390bf3
Fix flaky restartAfterRequestTimerExpires tests (#1084) 
Add wait to allow subscriptions to start.
This eliminates flakiness of tests restartAfterRequestTimerExpiresWhenNotGettingRecordsAfterInitialization() and restartAfterRequestTimerExpiresWhenInitialTaskExecutionIsRejected().
 2023-03-24 04:53:44 -07:00
noahbt
10cdf43b9d
Updated versions to SNAPSHOT (#1085) 2023-03-23 13:54:50 -04:00
Ryan French
2ecd1c4ac5
Allow tags to be added when lease table is created (#1065) 
* Allow tags to be added when lease table is created

* Add javadoc comment to new member variable

* DRY up creation of the CreateTable Request builder

* Fix compiler error

* Remove unnecessary eq functions

* Fix indentation

* Add patch
 2023-03-23 13:26:07 -04:00
furq-aws
0627ba50bb
Fix flaky HashRangesAreAlwaysComplete test (#1066) 
Updates depth value for Merge and Reshard with some In-Progress Parents tests to prevent invalid hierarchy trees.
 2023-03-22 09:33:28 -07:00
noahbt
b894669bda
Changes made for 2.4.8 (#1083) 2023-03-21 17:17:12 -07:00
stair
6be92dc4ef
Added metrics in ShutdownTask for scenarios when parent leases are missing. (#1080) 
+ optimizations in `ShutdownTask` (e.g., `Random` static instance,
eliminated over-used Function)
+ DRY+KISS on `ShutdownTaskTest`
+ deleted some dead code
 2023-03-21 19:52:17 -04:00
noahbt
177303d557
Changelog updated for 2.4.7 (#1078) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:21:54 -07:00
noahbt
26a692530d
Revert changes to pom property (#1077) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-20 16:03:36 -07:00
furq-aws
9d07af403d
Fix flaky InitializationWaitsWhenLeaseTableIsEmpty test (#1069) 
Update misconfigured Scheduler MIN_WAIT_TIME_FOR_LEASE_TABLE_CHECK_MILLIS test value to be in sync with source value
 2023-03-20 14:00:05 -07:00
noahbt
0d5007c04c
Files updated for KCL release 2.4.7 (#1067) 
Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-17 13:21:13 -07:00
dependabot[bot]
9ff99f0182
Bump awssdk.version from 2.19.31 to 2.20.8 (#1047) 
Bumps `awssdk.version` from 2.19.31 to 2.20.8.

Updates `kinesis` from 2.19.31 to 2.20.8

Updates `dynamodb` from 2.19.31 to 2.20.8

Updates `cloudwatch` from 2.19.31 to 2.20.8

Updates `netty-nio-client` from 2.19.31 to 2.20.8

Updates `sts` from 2.19.31 to 2.20.8

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:18:31 -07:00
dependabot[bot]
bc71990bec
Bump maven-javadoc-plugin from 3.3.1 to 3.5.0 (#1046) 
Bumps [maven-javadoc-plugin](https://github.com/apache/maven-javadoc-plugin) from 3.3.1 to 3.5.0.
- [Release notes](https://github.com/apache/maven-javadoc-plugin/releases)
- [Commits](https://github.com/apache/maven-javadoc-plugin/compare/maven-javadoc-plugin-3.3.1...maven-javadoc-plugin-3.5.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-javadoc-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:14:31 -07:00
dependabot[bot]
7416f8b915
Bump gsr.version from 1.1.13 to 1.1.14 (#1038) 
Bumps `gsr.version` from 1.1.13 to 1.1.14.

Updates `schema-registry-serde` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

Updates `schema-registry-common` from 1.1.13 to 1.1.14
- [Release notes](https://github.com/awslabs/aws-glue-schema-registry/releases)
- [Changelog](https://github.com/awslabs/aws-glue-schema-registry/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/aws-glue-schema-registry/compare/v1.1.13...v1.1.14)

---
updated-dependencies:
- dependency-name: software.amazon.glue:schema-registry-serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.glue:schema-registry-common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:11:05 -07:00
dependabot[bot]
0a3fcb07bd
Bump aws-java-sdk.version from 1.12.370 to 1.12.405 (#1037) 
Bumps `aws-java-sdk.version` from 1.12.370 to 1.12.405.

Updates `aws-java-sdk-core` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

Updates `aws-java-sdk-sts` from 1.12.370 to 1.12.405
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.12.370...1.12.405)

---
updated-dependencies:
- dependency-name: com.amazonaws:aws-java-sdk-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: com.amazonaws:aws-java-sdk-sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-15 14:09:46 -07:00
chenylee-aws
0cbd74f6e7
Allow leader to learn new leases upon re-election to avoid unnecessary shardSyncs (#1063) 2023-03-15 12:34:32 -07:00
noahbt
04a121a811
Add new metric to be emitted on lease creation (#1060) 
* Add new metric to be emitted on lease creation

* Rebase changes from master

---------

Co-authored-by: Noah Thomas <noahbt@amazon.com>
 2023-03-14 14:09:47 -07:00
stair
5bbb9768b5
DRY: simplification of HierarchicalShardSyncerTest. (#1059) 2023-03-13 20:05:40 -04:00
stair
c4204002af
Fixed retry storm in PrefetchRecordsPublisher. (#1062) 
+ DRY in `PrefetchRecordsPublisherTest`
 2023-03-13 19:57:01 -04:00
stair
504ea10859
Fixed NPE in LeaseCleanupManager. (#1061) 2023-03-10 15:07:46 -05:00
Abhit Sawwalakhe
27b166c5aa
Clean up in-memory state of deleted kinesis stream in MultiStreamMode (#1056) 
Co-authored-by: Abhit Sawwalakhe <sawwa@amazon.com>
 2023-03-08 13:39:35 -08:00
stair
43d43653d0
Documentation: added <pre> tags so fixed-format diagrams aren't garbled. (#1058) 
No functional change.
 2023-03-07 15:36:25 -05:00
stair
87dc586e6d
Added more logging in Scheduler w.r.t. StreamConfigs. (#1057) 2023-03-06 17:05:34 -05:00
stair
cd80c93966
Exposed convenience method of ExtendedSequenceNumber#isSentinelCheckpoint() (#1053) 
+ fixed unrelated parameterized log message in `ShardSyncTaskManager`
 2023-02-27 10:15:30 -08:00
stair
d8aa784f17
Removed a .swp file, and updated .gitignore. (#1043) 2023-02-27 10:02:36 -08:00
ZeyuLi-AWS
70cfc7d7ce
Fixed duplication of project version in children pom.xml (#1045) 2023-02-20 15:53:57 -08:00
stair
1c8bd8e71e
Minor optimizations (e.g., calculate-once, put instead of get+put) (#1041) 
and code cleanup (e.g., removed unused imports, updated Javadoc).

No functional change.
 2023-02-13 10:58:02 -08:00
stair
5bfd1ab289
Release Note updates to avoid duplication and bitrot (e.g., 1.x release (#1035) 
notes are not ported forward to 2.x).
 2023-02-13 13:20:54 -05:00
stair
4d94efac8f
Optimization: 9~15% improvement in KinesisDataFetcher wall-time after (#1034) 
converting `AWSExceptionManger` to a static variable.
 2023-02-13 13:16:28 -05:00
pelaezryan
9fb58a22bf
Increased logging verbosity around lease management. Also included additional javadocs for methods (#1040) 
Co-authored-by: Ryan Pelaez <rmpelaez@amazon.com>
 2023-02-13 12:28:54 -05:00
dependabot[bot]
34f19c5a7b
Bump rxjava from 3.1.5 to 3.1.6 (#1032) 
Bumps [rxjava](https://github.com/ReactiveX/RxJava) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/ReactiveX/RxJava/releases)
- [Commits](https://github.com/ReactiveX/RxJava/compare/v3.1.5...v3.1.6)

---
updated-dependencies:
- dependency-name: io.reactivex.rxjava3:rxjava
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-07 12:20:31 -05:00
dependabot[bot]
af71d9e224
Bump awssdk.version from 2.19.2 to 2.19.31 (#1030) 
Bumps `awssdk.version` from 2.19.2 to 2.19.31.

Updates `kinesis` from 2.19.2 to 2.19.31

Updates `dynamodb` from 2.19.2 to 2.19.31

Updates `cloudwatch` from 2.19.2 to 2.19.31

Updates `netty-nio-client` from 2.19.2 to 2.19.31

Updates `sts` from 2.19.2 to 2.19.31

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:kinesis
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:dynamodb
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:cloudwatch
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:netty-nio-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-07 12:19:15 -05:00
dependabot[bot]
5715e944e2
Bump slf4j-api from 2.0.0 to 2.0.6 (#1029) 
Bumps [slf4j-api](https://github.com/qos-ch/slf4j) from 2.0.0 to 2.0.6.
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/compare/v_2.0.0...v_2.0.6)

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-02-07 12:18:53 -05:00
stair
17b82a0e67
Refactored MultiStreamTracker to provide and enhance OOP for both (#1028) 
single- and multi-stream trackers.

+ converted `Scheduler#currentStreamConfigMap` to `ConcurrentHashMap`
+ eliminated a responsibility from Scheduler (i.e., orphan config generation)
 2023-02-06 23:04:34 -08:00
stair
dd429a2b1c
Removed CHECKSTYLE:OFF toggles which can invite/obscure sub-par code. (#1027) 
+ removed unused `assertHashRangeOfClosedShardIsCovered(...)` method
 2023-01-31 22:39:13 -08:00
JumpeiAnzai
6cb0100163
Fixing log message spacing (#795) 2023-01-31 18:36:12 -05:00
Ben Iofel
0535193394
Fix warning message typos (#956) 
Same as #875
 2023-01-31 18:26:14 -05:00
dependabot[bot]
28cb185e9c
Bump protobuf-java from 3.21.5 to 3.21.12 (#1015) 
Bumps [protobuf-java](https://github.com/protocolbuffers/protobuf) from 3.21.5 to 3.21.12.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/generate_changelog.py)
- [Commits](https://github.com/protocolbuffers/protobuf/compare/v3.21.5...v3.21.12)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-31 14:47:32 -05:00
Jan Sochor
65c95ed872
Pass isAtShardEnd correctly to processRecords call (#935) 
The default is false otherwise, i.e., the processor is always getting isAtShardEnd=false.
 2023-01-31 12:53:45 -05:00
CF
6146ff9851
Typo in Comment (#740) 
Implemetation -> Implementation
 2023-01-31 12:32:10 -05:00
stair
4411a3dc77
Added logging w.r.t. StreamConfig handling. (#1024) 2023-01-30 17:41:11 -05:00
Chenyuan Lee
0e86089123
Preparation for v2.4.5 (#1021) 2023-01-05 08:25:17 -08:00
Chenyuan Lee
17d0940f5d
Use AFTER_SEQUENCE_NUMBER iterator type for expired iterator request (#1014) 
* Use AFTER_SEQUENCE_NUMBER iterator type for expired iterator request

* Update Java docs and minor refactoring

* Fix Java doc

Co-authored-by: Chenyuan Lee <chenylee@amazon.com>
 2023-01-03 15:10:31 -08:00
Yu Zeng
676bb86b8e Preparation for v2.4.4 (Revise CHANGELOG/README) 2022-12-23 16:13:21 -08:00
Yu Zeng
adb3990481 Correct the KCL version in the main pom 2022-12-23 16:13:21 -08:00
Yu Zeng
780ca178d5 Preparation for v2.4.4 2022-12-23 15:52:58 -08:00
Yu Zeng
b1cc48af50 Upgrade aws sdk to the latest version 2022-12-23 15:52:58 -08:00
Yu Zeng
05ed537572 Preparation for v2.4.3 (revise changelog) 2022-09-06 15:34:32 -07:00
Yu Zeng
3580b082cd Upgrade awssdk to the latest version 2022-09-06 14:50:50 -07:00
Yu Zeng
5ac021e0c6 Upgrade more dependencies 2022-09-06 14:50:50 -07:00
dependabot[bot]
0e14dcfb2c Bump jcommander from 1.81 to 1.82 
Bumps [jcommander](https://github.com/cbeust/jcommander) from 1.81 to 1.82.
- [Release notes](https://github.com/cbeust/jcommander/releases)
- [Changelog](https://github.com/cbeust/jcommander/blob/master/CHANGELOG.md)
- [Commits](https://github.com/cbeust/jcommander/compare/1.81...1.82)

---
updated-dependencies:
- dependency-name: com.beust:jcommander
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-09-06 13:47:59 -07:00