Compare commits

...

7 commits
master ... v2.x

Author SHA1 Message Date
vincentvilo-aws
be89e7cf98
Reintroduce -SNAPSHOT version classifier (#1448) 2025-03-13 10:25:04 -07:00
vincentvilo-aws
7ff585eb61
Update documentation for upgrading to KCL v2.7.x (#1446) 2025-03-12 10:29:02 -07:00
vincentvilo-aws
6ba37b19bf
Prepare for release v2.7.0 (#1445) 2025-03-11 11:17:36 -07:00
vincentvilo-aws
4e2bc77fd1
Remove SDK v1 from KCL v2 (#1444)
* update dependabot to check for dependencies in v2.x

* exclude v1 sdk from glue dependency

* remove sdk v1 dependency from multilang pom.xml

* remove dependency on Regions class from sdk v1

* remove sdk v1 dependencies

* update docs for credential provider

* update documentation for NestedPropertyProcessor

* update KclStsAssumeRoleCredentialsProvider to have resolveCredentials()

* fix casing within AwsCredentialsProviderPropertyValueDecoderTest

* update unit tests to have endpointRegion

* update docs with endpointRegion
2025-03-10 16:20:38 -07:00
lucienlu-aws
08c874ad53
Update README notices (#1429) 2025-01-03 15:53:35 -08:00
vincentvilo-aws
da34720eda
Reintroduce -SNAPSHOT version classifier (#1412) 2024-12-13 14:00:46 -08:00
vincentvilo-aws
c69ccb6a80
Prepare for v2.6.1 (#1411) 2024-12-13 09:34:53 -08:00
24 changed files with 651 additions and 522 deletions

View file

@ -1,11 +1,22 @@
version: 2
updates:
# branch - master
- package-ecosystem: "maven"
directory: "/"
labels:
- "dependencies"
- "v3.x"
schedule:
interval: "weekly"
# branch - v2.x
- package-ecosystem: "maven"
directory: "/"
labels:
- "dependencies"
- "v2.x"
schedule:
target-branch: "v2.x"
schedule:
interval: "weekly"
# branch - v1.x

View file

@ -3,6 +3,34 @@
For **1.x** release notes, please see [v1.x/CHANGELOG.md](https://github.com/awslabs/amazon-kinesis-client/blob/v1.x/CHANGELOG.md)
---
### Release 2.7.0 (2025-03-11)
* [#1444](https://github.com/awslabs/amazon-kinesis-client/pull/1444) Fully remove dependency on the AWS SDK for Java 1.x which will reach [end-of-support by December 31st, 2025](https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-java-v1-x-on-december-31-2025/).
* The Glue Schema Registry integration functionality no longer depends on AWS SDK for Java 1.x. Previously, it required this as a transient dependency.
* Multilangdaemon has been upgraded to use AWS SDK for Java 2.x. It no longer depends on AWS SDK for Java 1.x.
### Release 2.6.1 (2024-12-13)
* [#1365](https://github.com/awslabs/amazon-kinesis-client/pull/1365) Add config to enable point in time recovery for the DynamoDB lease table created by KCL
* [#1325](https://github.com/awslabs/amazon-kinesis-client/pull/1325) Add logic to sync from lease table on first app bootstrap
* [#1372](https://github.com/awslabs/amazon-kinesis-client/pull/1372) Fix debug and trace level logging to log all worker loops
* [#1319](https://github.com/awslabs/amazon-kinesis-client/pull/1319) Fix a race condition between ShardConsumer shutdown and initialization
* [#1354](https://github.com/awslabs/amazon-kinesis-client/pull/1354) Fix bug where the in-memory lease is updated even when DDB fails to update
* [#1363](https://github.com/awslabs/amazon-kinesis-client/pull/1363) Fix some of the warnings emitted by maven during build
* [#1358](https://github.com/awslabs/amazon-kinesis-client/pull/1358) Remove ShutdownNotificationAware and update javadocs
* [#1329](https://github.com/awslabs/amazon-kinesis-client/pull/1329) Remove unnecessary lambda
* [#1340](https://github.com/awslabs/amazon-kinesis-client/pull/1340) Generate wrappers from proto files instead of shipping them directly
* [#1332](https://github.com/awslabs/amazon-kinesis-client/pull/1332) Enforce consistent code style with palantir-java-format
* [#1294](https://github.com/awslabs/amazon-kinesis-client/pull/1294) Upgrade org.apache.maven.plugins:maven-checkstyle-plugin from 3.3.0 to 3.3.1
* [#1296](https://github.com/awslabs/amazon-kinesis-client/pull/1296) Upgrade org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.5
* [#1312](https://github.com/awslabs/amazon-kinesis-client/pull/1312) Upgrade commons-io:commons-io from 2.15.1 to 2.16.1
* [#1331](https://github.com/awslabs/amazon-kinesis-client/pull/1331) Upgrade maven-compiler-plugin from 3.11.0 to 3.13.0
* [#1335](https://github.com/awslabs/amazon-kinesis-client/pull/1335) Upgrade org.slf4j:slf4j-api from 2.0.7 to 2.0.13
* [#1336](https://github.com/awslabs/amazon-kinesis-client/pull/1336) Upgrade io.reactivex.rxjava3:rxjava from 3.1.6 to 3.1.8
* [#1337](https://github.com/awslabs/amazon-kinesis-client/pull/1337) Upgrade com.google.protobuf:protobuf-java from 3.21.12 to 4.27.0
* [#1341](https://github.com/awslabs/amazon-kinesis-client/pull/1341) Upgrade awssdk.version from 2.25.11 to 2.25.64
* [#1342](https://github.com/awslabs/amazon-kinesis-client/pull/1342) Upgrade org.apache.maven.plugins:maven-javadoc-plugin from 3.6.3 to 3.7.0
* [#1343](https://github.com/awslabs/amazon-kinesis-client/pull/1343) Upgrade org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.5
* [#1383](https://github.com/awslabs/amazon-kinesis-client/pull/1383) Upgrade com.google.protobuf:protobuf-java from 4.27.0 to 4.27.5
### Release 2.6.0 (2024-05-01)
* [#1317](https://github.com/awslabs/amazon-kinesis-client/pull/1317) Add enablePriorityLeaseAssignment config
* [#1320](https://github.com/awslabs/amazon-kinesis-client/pull/1320) Update lease taker to get unassigned leases

View file

@ -10,16 +10,6 @@ The **Amazon Kinesis Client Library for Java** (Amazon KCL) enables Java develop
* [KCL Documentation](docs/) (folder)
* [Issues][kinesis-client-library-issues]
### Recommended Upgrade for All Users of the 1.x Amazon Kinesis Client
:warning: We recommend customers to migrate to 1.14.1 or newer to avoid [known bugs](https://github.com/awslabs/amazon-kinesis-client/issues/778) in 1.14.0 version
### Recommended Upgrade for All Users of the 2.x Amazon Kinesis Client
**:warning: It's highly recommended for users of version 2.0 of the Amazon Kinesis Client to upgrade to version 2.0.3 or later. A [bug has been](https://github.com/awslabs/amazon-kinesis-client/issues/391) identified in versions prior to 2.0.3 that could cause records to be delivered to the wrong record processor.**
**:information_source: Amazon Kinesis Client versions 1.x are not impacted.**
Please open an issue if you have any questions.
## Features
* Provides an easy-to-use programming model for processing data using Amazon Kinesis
@ -58,12 +48,22 @@ To make it easier for developers to write record processors in other languages,
## Using the KCL
The recommended way to use the KCL for Java is to consume it from Maven.
## 🚨Important: Migration to KCL 2.7 or later with MultiLangDaemon - Credential Provider Changes Required
KCL version 2.7.0 and later uses AWS SDK for Java 2.x instead of AWS SDK for Java 1.x. All MultiLangDaemon users
upgrading from earlier versions must update their credential provider configuration in the `.properties` file to use
credentials provider name for AWS SDK for Java 2.x. Failure to do this will cause your multilang KCL application to fail
during startup with credential provider construction errors. Please check the following link for the credentials
provider mapping and MultiLangDaemon credentials provider configuration guide.
- [AWS SDK for Java 1.x to 2.x Credentials Provider Mapping](aws.amazon.com/sdk-for-java/latest/developer-guide/migration-client-credentials.html#credentials-changes-mapping)
- [KCL Multilang Credentials Provider Configuration Guide](https://github.com/aws/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md)
### Version 2.x
``` xml
<dependency>
<groupId>software.amazon.kinesis</groupId>
<artifactId>amazon-kinesis-client</artifactId>
<version>2.6.0</version>
<version>2.7.0</version>
</dependency>
```
@ -84,6 +84,10 @@ The recommended way to use the KCL for Java is to consume it from Maven.
| 2.x | [master/CHANGELOG.md](CHANGELOG.md) |
| 1.x | [v1.x/CHANGELOG.md](https://github.com/awslabs/amazon-kinesis-client/blob/v1.x/CHANGELOG.md) |
## Notices
We recommend all users to migrate to the latest respective versions to avoid known issues and benefit from all improvements.
[docs-signup]: http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-setup.html
[kcl-javadoc]: https://javadoc.io/doc/software.amazon.kinesis/amazon-kinesis-client/
[kinesis]: http://aws.amazon.com/kinesis

View file

@ -21,16 +21,12 @@
<parent>
<artifactId>amazon-kinesis-client-pom</artifactId>
<groupId>software.amazon.kinesis</groupId>
<version>2.6.1-SNAPSHOT</version>
<version>2.7.1-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>amazon-kinesis-client-multilang</artifactId>
<properties>
<aws-java-sdk.version>1.12.668</aws-java-sdk.version>
</properties>
<dependencies>
<dependency>
<groupId>software.amazon.kinesis</groupId>
@ -43,32 +39,6 @@
<version>${awssdk.version}</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>${aws-java-sdk.version}</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
<exclusion>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-cbor</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sts</artifactId>
<version>${aws-java-sdk.version}</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>

View file

@ -61,10 +61,10 @@ import software.amazon.kinesis.coordinator.Scheduler;
* applicationName = PythonKCLSample
*
* # Users can change the credentials provider the KCL will use to retrieve credentials.
* # The DefaultAWSCredentialsProviderChain checks several other providers, which is
* # The DefaultCredentialsProvider checks several other providers, which is
* # described here:
* # http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html
* AWSCredentialsProvider = DefaultAWSCredentialsProviderChain
* # https://sdk.amazonaws.com/java/api/2.0.0-preview-11/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html
* AwsCredentialsProvider = DefaultCredentialsProvider
* </pre>
*/
@Slf4j

View file

@ -15,13 +15,14 @@
package software.amazon.kinesis.multilang;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.amazonaws.regions.Regions;
import com.google.common.base.CaseFormat;
import lombok.AccessLevel;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import software.amazon.awssdk.regions.Region;
/**
* Key-Value pairs which may be nested in, and extracted from, a property value
@ -73,8 +74,13 @@ public enum NestedPropertyKey {
* @see <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions">Available Regions</a>
*/
ENDPOINT_REGION {
void visit(final NestedPropertyProcessor processor, final String region) {
processor.acceptEndpointRegion(Regions.fromName(region));
void visit(final NestedPropertyProcessor processor, final String regionName) {
List<Region> validRegions = Region.regions();
Region region = Region.of(regionName);
if (!validRegions.contains(region)) {
throw new IllegalArgumentException("Invalid region name: " + regionName);
}
processor.acceptEndpointRegion(region);
}
},

View file

@ -14,7 +14,7 @@
*/
package software.amazon.kinesis.multilang;
import com.amazonaws.regions.Regions;
import software.amazon.awssdk.regions.Region;
/**
* Defines methods to process {@link NestedPropertyKey}s.
@ -26,11 +26,11 @@ public interface NestedPropertyProcessor {
*
* @param serviceEndpoint the service endpoint either with or without the protocol
* (e.g., https://sns.us-west-1.amazonaws.com, sns.us-west-1.amazonaws.com)
* @param signingRegion the region to use for SigV4 signing of requests (e.g. us-west-1)
* @param signingRegion the region to use for the client (e.g. us-west-1)
*
* @see #acceptEndpointRegion(Regions)
* @see <a href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/client/builder/AwsClientBuilder.EndpointConfiguration.html">
* AwsClientBuilder.EndpointConfiguration</a>
* @see #acceptEndpointRegion(Region)
* @see <a href="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/client/builder/SdkClientBuilder.html#endpointOverride(java.net.URI)">
* AwsClientBuilder.endpointOverride</a>
*/
void acceptEndpoint(String serviceEndpoint, String signingRegion);
@ -42,7 +42,7 @@ public interface NestedPropertyProcessor {
*
* @see #acceptEndpoint(String, String)
*/
void acceptEndpointRegion(Regions region);
void acceptEndpointRegion(Region region);
/**
* Set the external id, an optional field to designate who can assume an IAM role.

View file

@ -1,86 +0,0 @@
/*
* Copyright 2023 Amazon.com, Inc. or its affiliates.
* Licensed under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package software.amazon.kinesis.multilang.auth;
import java.util.Arrays;
import com.amazonaws.auth.AWSSessionCredentials;
import com.amazonaws.auth.AWSSessionCredentialsProvider;
import com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider;
import com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.Builder;
import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.securitytoken.AWSSecurityTokenService;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient;
import software.amazon.kinesis.multilang.NestedPropertyKey;
import software.amazon.kinesis.multilang.NestedPropertyProcessor;
/**
* An {@link AWSSessionCredentialsProvider} that is backed by STSAssumeRole.
*/
public class KclSTSAssumeRoleSessionCredentialsProvider
implements AWSSessionCredentialsProvider, NestedPropertyProcessor {
private final Builder builder;
private final STSAssumeRoleSessionCredentialsProvider provider;
/**
*
* @param params vararg parameters which must include roleArn at index=0,
* and roleSessionName at index=1
*/
public KclSTSAssumeRoleSessionCredentialsProvider(final String[] params) {
this(params[0], params[1], Arrays.copyOfRange(params, 2, params.length));
}
public KclSTSAssumeRoleSessionCredentialsProvider(
final String roleArn, final String roleSessionName, final String... params) {
builder = new Builder(roleArn, roleSessionName);
NestedPropertyKey.parse(this, params);
provider = builder.build();
}
@Override
public AWSSessionCredentials getCredentials() {
return provider.getCredentials();
}
@Override
public void refresh() {
// do nothing
}
@Override
public void acceptEndpoint(final String serviceEndpoint, final String signingRegion) {
final EndpointConfiguration endpoint = new EndpointConfiguration(serviceEndpoint, signingRegion);
final AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClient.builder()
.withEndpointConfiguration(endpoint)
.build();
builder.withStsClient(stsClient);
}
@Override
public void acceptEndpointRegion(final Regions region) {
final AWSSecurityTokenService stsClient =
AWSSecurityTokenServiceClient.builder().withRegion(region).build();
builder.withStsClient(stsClient);
}
@Override
public void acceptExternalId(final String externalId) {
builder.withExternalId(externalId);
}
}

View file

@ -0,0 +1,75 @@
/*
* Copyright 2024 Amazon.com, Inc. or its affiliates.
* Licensed under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package software.amazon.kinesis.multilang.auth;
import java.net.URI;
import java.util.Arrays;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.sts.StsClient;
import software.amazon.awssdk.services.sts.StsClientBuilder;
import software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider;
import software.amazon.awssdk.services.sts.model.AssumeRoleRequest;
import software.amazon.awssdk.services.sts.model.AssumeRoleRequest.Builder;
import software.amazon.kinesis.multilang.NestedPropertyKey;
import software.amazon.kinesis.multilang.NestedPropertyProcessor;
public class KclStsAssumeRoleCredentialsProvider implements AwsCredentialsProvider, NestedPropertyProcessor {
private final Builder assumeRoleRequestBuilder;
private final StsClientBuilder stsClientBuilder;
private final StsAssumeRoleCredentialsProvider stsAssumeRoleCredentialsProvider;
public KclStsAssumeRoleCredentialsProvider(String[] params) {
this(params[0], params[1], Arrays.copyOfRange(params, 2, params.length));
}
public KclStsAssumeRoleCredentialsProvider(String roleArn, String roleSessionName, String... params) {
this.assumeRoleRequestBuilder =
AssumeRoleRequest.builder().roleArn(roleArn).roleSessionName(roleSessionName);
this.stsClientBuilder = StsClient.builder();
NestedPropertyKey.parse(this, params);
this.stsAssumeRoleCredentialsProvider = StsAssumeRoleCredentialsProvider.builder()
.refreshRequest(assumeRoleRequestBuilder.build())
.asyncCredentialUpdateEnabled(true)
.stsClient(stsClientBuilder.build())
.build();
}
@Override
public AwsCredentials resolveCredentials() {
return stsAssumeRoleCredentialsProvider.resolveCredentials();
}
@Override
public void acceptEndpoint(String serviceEndpoint, String signingRegion) {
if (!serviceEndpoint.startsWith("http://") && !serviceEndpoint.startsWith("https://")) {
serviceEndpoint = "https://" + serviceEndpoint;
}
stsClientBuilder.endpointOverride(URI.create(serviceEndpoint));
stsClientBuilder.region(Region.of(signingRegion));
}
@Override
public void acceptEndpointRegion(Region region) {
stsClientBuilder.region(region);
}
@Override
public void acceptExternalId(String externalId) {
assumeRoleRequestBuilder.externalId(externalId);
}
}

View file

@ -1,182 +0,0 @@
/*
* Copyright 2019 Amazon.com, Inc. or its affiliates.
* Licensed under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package software.amazon.kinesis.multilang.config;
import java.lang.reflect.InvocationTargetException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSCredentialsProviderChain;
import lombok.extern.slf4j.Slf4j;
/**
* Get AWSCredentialsProvider property.
*/
@Slf4j
class AWSCredentialsProviderPropertyValueDecoder implements IPropertyValueDecoder<AWSCredentialsProvider> {
private static final String LIST_DELIMITER = ",";
private static final String ARG_DELIMITER = "|";
/**
* Constructor.
*/
AWSCredentialsProviderPropertyValueDecoder() {}
/**
* Get AWSCredentialsProvider property.
*
* @param value
* property value as String
* @return corresponding variable in correct type
*/
@Override
public AWSCredentialsProvider decodeValue(String value) {
if (value != null) {
List<String> providerNames = getProviderNames(value);
List<AWSCredentialsProvider> providers = getValidCredentialsProviders(providerNames);
AWSCredentialsProvider[] ps = new AWSCredentialsProvider[providers.size()];
providers.toArray(ps);
return new AWSCredentialsProviderChain(providers);
} else {
throw new IllegalArgumentException("Property AWSCredentialsProvider is missing.");
}
}
/**
* @return list of supported types
*/
@Override
public List<Class<AWSCredentialsProvider>> getSupportedTypes() {
return Collections.singletonList(AWSCredentialsProvider.class);
}
/**
* Convert string list to a list of valid credentials providers.
*/
private static List<AWSCredentialsProvider> getValidCredentialsProviders(List<String> providerNames) {
List<AWSCredentialsProvider> credentialsProviders = new ArrayList<>();
for (String providerName : providerNames) {
final String[] nameAndArgs = providerName.split("\\" + ARG_DELIMITER);
final Class<? extends AWSCredentialsProvider> clazz;
try {
final Class<?> c = Class.forName(nameAndArgs[0]);
if (!AWSCredentialsProvider.class.isAssignableFrom(c)) {
continue;
}
clazz = (Class<? extends AWSCredentialsProvider>) c;
} catch (ClassNotFoundException cnfe) {
// Providers are a product of prefixed Strings to cover multiple
// namespaces (e.g., "Foo" -> { "some.auth.Foo", "kcl.auth.Foo" }).
// It's expected that many class names will not resolve.
continue;
}
log.info("Attempting to construct {}", clazz);
AWSCredentialsProvider provider = null;
if (nameAndArgs.length > 1) {
final String[] varargs = Arrays.copyOfRange(nameAndArgs, 1, nameAndArgs.length);
// attempt to invoke an explicit N-arg constructor of FooClass(String, String, ...)
provider = constructProvider(providerName, () -> {
Class<?>[] argTypes = new Class<?>[nameAndArgs.length - 1];
Arrays.fill(argTypes, String.class);
return clazz.getConstructor(argTypes).newInstance(varargs);
});
if (provider == null) {
// attempt to invoke a public varargs/array constructor of FooClass(String[])
provider = constructProvider(providerName, () -> clazz.getConstructor(String[].class)
.newInstance((Object) varargs));
}
}
if (provider == null) {
// regardless of parameters, fallback to invoke a public no-arg constructor
provider = constructProvider(providerName, clazz::newInstance);
}
if (provider != null) {
credentialsProviders.add(provider);
}
}
return credentialsProviders;
}
private static List<String> getProviderNames(String property) {
// assume list delimiter is ","
String[] elements = property.split(LIST_DELIMITER);
List<String> result = new ArrayList<>();
for (int i = 0; i < elements.length; i++) {
String string = elements[i].trim();
if (!string.isEmpty()) {
// find all possible names and add them to name list
result.addAll(getPossibleFullClassNames(string));
}
}
return result;
}
private static List<String> getPossibleFullClassNames(final String provider) {
return Stream.of(
// Customer provides a short name of common providers in com.amazonaws.auth package
// (e.g., any classes implementing the AWSCredentialsProvider interface)
// @see
// http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html
"com.amazonaws.auth.",
// Customer provides a short name of a provider offered by this multi-lang package
"software.amazon.kinesis.multilang.auth.",
// Customer provides a fully-qualified provider name, or a custom credentials provider
// (e.g., com.amazonaws.auth.ClasspathFileCredentialsProvider, org.mycompany.FooProvider)
"")
.map(prefix -> prefix + provider)
.collect(Collectors.toList());
}
@FunctionalInterface
private interface CredentialsProviderConstructor<T extends AWSCredentialsProvider> {
T construct()
throws IllegalAccessException, InstantiationException, InvocationTargetException, NoSuchMethodException;
}
/**
* Attempts to construct an {@link AWSCredentialsProvider}.
*
* @param providerName Raw, unmodified provider name. Should there be an
* Exeception during construction, this parameter will be logged.
* @param constructor supplier-like function that will perform the construction
* @return the constructed provider, if successful; otherwise, null
*
* @param <T> type of the CredentialsProvider to construct
*/
private static <T extends AWSCredentialsProvider> T constructProvider(
final String providerName, final CredentialsProviderConstructor<T> constructor) {
try {
return constructor.construct();
} catch (NoSuchMethodException ignored) {
// ignore
} catch (IllegalAccessException | InstantiationException | InvocationTargetException | RuntimeException e) {
log.warn("Failed to construct {}", providerName, e);
}
return null;
}
}

View file

@ -0,0 +1,261 @@
/*
* Copyright 2019 Amazon.com, Inc. or its affiliates.
* Licensed under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package software.amazon.kinesis.multilang.config;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import lombok.extern.slf4j.Slf4j;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider;
import software.amazon.kinesis.multilang.auth.KclStsAssumeRoleCredentialsProvider;
/**
* Get AwsCredentialsProvider property.
*/
@Slf4j
class AwsCredentialsProviderPropertyValueDecoder implements IPropertyValueDecoder<AwsCredentialsProvider> {
private static final String LIST_DELIMITER = ",";
private static final String ARG_DELIMITER = "|";
/**
* Constructor.
*/
AwsCredentialsProviderPropertyValueDecoder() {}
/**
* Get AwsCredentialsProvider property.
*
* @param value
* property value as String
* @return corresponding variable in correct type
*/
@Override
public AwsCredentialsProvider decodeValue(String value) {
if (value != null) {
List<String> providerNames = getProviderNames(value);
List<AwsCredentialsProvider> providers = getValidCredentialsProviders(providerNames);
AwsCredentialsProvider[] ps = new AwsCredentialsProvider[providers.size()];
providers.toArray(ps);
if (providers.isEmpty()) {
log.warn("Unable to construct any provider with name {}", value);
log.warn("Please verify that all AwsCredentialsProvider properties are passed correctly");
}
return AwsCredentialsProviderChain.builder()
.credentialsProviders(providers)
.build();
} else {
throw new IllegalArgumentException("Property AwsCredentialsProvider is missing.");
}
}
/**
* @return list of supported types
*/
@Override
public List<Class<AwsCredentialsProvider>> getSupportedTypes() {
return Collections.singletonList(AwsCredentialsProvider.class);
}
/**
* Convert string list to a list of valid credentials providers.
*/
private static List<AwsCredentialsProvider> getValidCredentialsProviders(List<String> providerNames) {
List<AwsCredentialsProvider> credentialsProviders = new ArrayList<>();
for (String providerName : providerNames) {
final String[] nameAndArgs = providerName.split("\\" + ARG_DELIMITER);
final Class<? extends AwsCredentialsProvider> clazz = getClass(nameAndArgs[0]);
if (clazz == null) {
continue;
}
log.info("Attempting to construct {}", clazz);
final String[] varargs =
nameAndArgs.length > 1 ? Arrays.copyOfRange(nameAndArgs, 1, nameAndArgs.length) : new String[0];
AwsCredentialsProvider provider = tryConstructor(providerName, clazz, varargs);
if (provider == null) {
provider = tryCreate(providerName, clazz, varargs);
}
if (provider != null) {
log.info("Provider constructed successfully: {}", provider);
credentialsProviders.add(provider);
}
}
return credentialsProviders;
}
private static AwsCredentialsProvider tryConstructor(
String providerName, Class<? extends AwsCredentialsProvider> clazz, String[] varargs) {
AwsCredentialsProvider provider =
constructProvider(providerName, () -> getConstructorWithVarArgs(clazz, varargs));
if (provider == null) {
provider = constructProvider(providerName, () -> getConstructorWithArgs(clazz, varargs));
}
if (provider == null) {
provider = constructProvider(providerName, clazz::newInstance);
}
return provider;
}
private static AwsCredentialsProvider tryCreate(
String providerName, Class<? extends AwsCredentialsProvider> clazz, String[] varargs) {
AwsCredentialsProvider provider =
constructProvider(providerName, () -> getCreateMethod(clazz, (Object) varargs));
if (provider == null) {
provider = constructProvider(providerName, () -> getCreateMethod(clazz, varargs));
}
if (provider == null) {
provider = constructProvider(providerName, () -> getCreateMethod(clazz));
}
return provider;
}
private static AwsCredentialsProvider getConstructorWithVarArgs(
Class<? extends AwsCredentialsProvider> clazz, String[] varargs) {
try {
return clazz.getConstructor(String[].class).newInstance((Object) varargs);
} catch (Exception e) {
return null;
}
}
private static AwsCredentialsProvider getConstructorWithArgs(
Class<? extends AwsCredentialsProvider> clazz, String[] varargs) {
try {
Class<?>[] argTypes = new Class<?>[varargs.length];
Arrays.fill(argTypes, String.class);
return clazz.getConstructor(argTypes).newInstance((Object[]) varargs);
} catch (Exception e) {
return null;
}
}
private static AwsCredentialsProvider getCreateMethod(
Class<? extends AwsCredentialsProvider> clazz, Object... args) {
try {
Class<?>[] argTypes = new Class<?>[args.length];
for (int i = 0; i < args.length; i++) {
argTypes[i] = args[i].getClass();
}
Method createMethod = clazz.getDeclaredMethod("create", argTypes);
if (Modifier.isStatic(createMethod.getModifiers())) {
return clazz.cast(createMethod.invoke(null, args));
} else {
log.warn("Found non-static create() method in {}", clazz.getName());
}
} catch (NoSuchMethodException e) {
// No matching create method found for class
} catch (Exception e) {
log.warn("Failed to invoke create() method in {}", clazz.getName(), e);
}
return null;
}
/**
* Resolves the class for the given provider name.
*
* @param providerName A string containing the provider name.
*
* @return The Class object representing the resolved AwsCredentialsProvider implementation,
* or null if the class cannot be resolved or does not extend AwsCredentialsProvider.
*/
private static Class<? extends AwsCredentialsProvider> getClass(String providerName) {
// Convert any form of StsAssumeRoleCredentialsProvider string to KclStsAssumeRoleCredentialsProvider
if (providerName.equals(StsAssumeRoleCredentialsProvider.class.getSimpleName())
|| providerName.equals(StsAssumeRoleCredentialsProvider.class.getName())) {
providerName = KclStsAssumeRoleCredentialsProvider.class.getName();
}
try {
final Class<?> c = Class.forName(providerName);
if (!AwsCredentialsProvider.class.isAssignableFrom(c)) {
return null;
}
return (Class<? extends AwsCredentialsProvider>) c;
} catch (ClassNotFoundException cnfe) {
// Providers are a product of prefixed Strings to cover multiple
// namespaces (e.g., "Foo" -> { "some.auth.Foo", "kcl.auth.Foo" }).
// It's expected that many class names will not resolve.
return null;
}
}
private static List<String> getProviderNames(String property) {
// assume list delimiter is ","
String[] elements = property.split(LIST_DELIMITER);
List<String> result = new ArrayList<>();
for (int i = 0; i < elements.length; i++) {
String string = elements[i].trim();
if (!string.isEmpty()) {
// find all possible names and add them to name list
result.addAll(getPossibleFullClassNames(string));
}
}
return result;
}
private static List<String> getPossibleFullClassNames(final String provider) {
return Stream.of(
// Customer provides a short name of a provider offered by this multi-lang package
"software.amazon.kinesis.multilang.auth.",
// Customer provides a short name of common providers in software.amazon.awssdk.auth.credentials
// package (e.g., any classes implementing the AwsCredentialsProvider interface)
// @see
// https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/AwsCredentialsProvider.html
"software.amazon.awssdk.auth.credentials.",
// Customer provides a fully-qualified provider name, or a custom credentials provider
// (e.g., org.mycompany.FooProvider)
"")
.map(prefix -> prefix + provider)
.collect(Collectors.toList());
}
@FunctionalInterface
private interface CredentialsProviderConstructor<T extends AwsCredentialsProvider> {
T construct()
throws IllegalAccessException, InstantiationException, InvocationTargetException, NoSuchMethodException;
}
/**
* Attempts to construct an {@link AwsCredentialsProvider}.
*
* @param providerName Raw, unmodified provider name. Should there be an
* Exception during construction, this parameter will be logged.
* @param constructor supplier-like function that will perform the construction
* @return the constructed provider, if successful; otherwise, null
*
* @param <T> type of the CredentialsProvider to construct
*/
private static <T extends AwsCredentialsProvider> T constructProvider(
final String providerName, final CredentialsProviderConstructor<T> constructor) {
try {
return constructor.construct();
} catch (NoSuchMethodException
| IllegalAccessException
| InstantiationException
| InvocationTargetException
| RuntimeException ignored) {
// ignore
}
return null;
}
}

View file

@ -28,7 +28,7 @@ import software.amazon.kinesis.common.StreamIdentifier;
/**
* KinesisClientLibConfigurator constructs a KinesisClientLibConfiguration from java properties file. The following
* three properties must be provided. 1) "applicationName" 2) "streamName" 3) "AWSCredentialsProvider"
* three properties must be provided. 1) "applicationName" 2) "streamName" 3) "AwsCredentialsProvider"
* KinesisClientLibConfigurator will help to automatically assign the value of "workerId" if this property is not
* provided. In the specified properties file, any properties, which matches the variable name in
* KinesisClientLibConfiguration and has a corresponding "with{variableName}" setter method, will be read in, and its
@ -62,7 +62,7 @@ public class KinesisClientLibConfigurator {
properties.entrySet().forEach(e -> {
try {
log.info("Processing (key={}, value={})", e.getKey(), e.getValue());
utilsBean.setProperty(configuration, (String) e.getKey(), e.getValue());
utilsBean.setProperty(configuration, processKey((String) e.getKey()), e.getValue());
} catch (IllegalAccessException | InvocationTargetException ex) {
throw new RuntimeException(ex);
}
@ -110,4 +110,17 @@ public class KinesisClientLibConfigurator {
}
return getConfiguration(properties);
}
/**
* Processes a configuration key to normalize AWS credentials provider naming. Necessary to conform to
* autogenerated setters.
* @param key the config param key
* @return case-configured param key name
*/
String processKey(String key) {
if (key.toLowerCase().startsWith("awscredentialsprovider")) {
key = key.replaceAll("(?i)awscredentialsprovider", "awsCredentialsProvider");
}
return key;
}
}

View file

@ -55,7 +55,6 @@ import software.amazon.kinesis.leases.ShardPrioritization;
import software.amazon.kinesis.lifecycle.LifecycleConfig;
import software.amazon.kinesis.metrics.MetricsConfig;
import software.amazon.kinesis.metrics.MetricsLevel;
import software.amazon.kinesis.multilang.config.credentials.V2CredentialWrapper;
import software.amazon.kinesis.processor.ProcessorConfig;
import software.amazon.kinesis.processor.ShardRecordProcessorFactory;
import software.amazon.kinesis.retrieval.RetrievalConfig;
@ -196,19 +195,19 @@ public class MultiLangDaemonConfiguration {
private final BuilderDynaBean kinesisCredentialsProvider;
public void setAWSCredentialsProvider(String providerString) {
public void setAwsCredentialsProvider(String providerString) {
kinesisCredentialsProvider.set("", providerString);
}
private final BuilderDynaBean dynamoDBCredentialsProvider;
public void setAWSCredentialsProviderDynamoDB(String providerString) {
public void setAwsCredentialsProviderDynamoDB(String providerString) {
dynamoDBCredentialsProvider.set("", providerString);
}
private final BuilderDynaBean cloudWatchCredentialsProvider;
public void setAWSCredentialsProviderCloudWatch(String providerString) {
public void setAwsCredentialsProviderCloudWatch(String providerString) {
cloudWatchCredentialsProvider.set("", providerString);
}
@ -282,9 +281,9 @@ public class MultiLangDaemonConfiguration {
ArrayConverter arrayConverter = new ArrayConverter(String[].class, new StringConverter());
arrayConverter.setDelimiter(',');
convertUtilsBean.register(arrayConverter, String[].class);
AWSCredentialsProviderPropertyValueDecoder oldCredentialsDecoder =
new AWSCredentialsProviderPropertyValueDecoder();
Function<String, ?> converter = s -> new V2CredentialWrapper(oldCredentialsDecoder.decodeValue(s));
AwsCredentialsProviderPropertyValueDecoder credentialsDecoder =
new AwsCredentialsProviderPropertyValueDecoder();
Function<String, ?> converter = credentialsDecoder::decodeValue;
this.kinesisCredentialsProvider = new BuilderDynaBean(
AwsCredentialsProvider.class, convertUtilsBean, converter, CREDENTIALS_DEFAULT_SEARCH_PATH);

View file

@ -1,52 +0,0 @@
/*
* Copyright 2019 Amazon.com, Inc. or its affiliates.
* Licensed under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package software.amazon.kinesis.multilang.config.credentials;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSSessionCredentials;
import lombok.RequiredArgsConstructor;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsSessionCredentials;
@RequiredArgsConstructor
public class V2CredentialWrapper implements AwsCredentialsProvider {
private final AWSCredentialsProvider oldCredentialsProvider;
@Override
public AwsCredentials resolveCredentials() {
AWSCredentials current = oldCredentialsProvider.getCredentials();
if (current instanceof AWSSessionCredentials) {
return AwsSessionCredentials.create(
current.getAWSAccessKeyId(),
current.getAWSSecretKey(),
((AWSSessionCredentials) current).getSessionToken());
}
return new AwsCredentials() {
@Override
public String accessKeyId() {
return current.getAWSAccessKeyId();
}
@Override
public String secretAccessKey() {
return current.getAWSSecretKey();
}
};
}
}

View file

@ -65,7 +65,7 @@ public class MultiLangDaemonConfigTest {
String properties = String.format(
"executableName = %s\n"
+ "applicationName = %s\n"
+ "AWSCredentialsProvider = DefaultAWSCredentialsProviderChain\n"
+ "AwsCredentialsProvider = DefaultCredentialsProvider\n"
+ "processingLanguage = malbolge\n"
+ "regionName = %s\n",
EXE, APPLICATION_NAME, "us-east-1");
@ -182,7 +182,7 @@ public class MultiLangDaemonConfigTest {
@Test
public void testPropertyValidation() {
String propertiesNoExecutableName = "applicationName = testApp \n" + "streamName = fakeStream \n"
+ "AWSCredentialsProvider = DefaultAWSCredentialsProviderChain\n" + "processingLanguage = malbolge";
+ "AwsCredentialsProvider = DefaultCredentialsProvider\n" + "processingLanguage = malbolge";
ClassLoader classLoader = Mockito.mock(ClassLoader.class);
Mockito.doReturn(new ByteArrayInputStream(propertiesNoExecutableName.getBytes()))

View file

@ -14,11 +14,11 @@
*/
package software.amazon.kinesis.multilang;
import com.amazonaws.regions.Regions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;
import software.amazon.awssdk.regions.Region;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.verify;
@ -64,9 +64,9 @@ public class NestedPropertyKeyTest {
@Test
public void testEndpointRegion() {
final Regions expectedRegion = Regions.GovCloud;
final Region expectedRegion = Region.US_GOV_WEST_1;
parse(mockProcessor, createKey(ENDPOINT_REGION, expectedRegion.getName()));
parse(mockProcessor, createKey(ENDPOINT_REGION, expectedRegion.id()));
verify(mockProcessor).acceptEndpointRegion(expectedRegion);
}

View file

@ -20,7 +20,7 @@ import org.junit.Test;
import static org.junit.Assert.assertEquals;
public class KclSTSAssumeRoleSessionCredentialsProviderTest {
public class KclStsAssumeRoleCredentialsProviderTest {
private static final String ARN = "arn";
private static final String SESSION_NAME = "sessionName";
@ -31,29 +31,29 @@ public class KclSTSAssumeRoleSessionCredentialsProviderTest {
*/
@Test
public void testConstructorWithoutOptionalParams() {
new KclSTSAssumeRoleSessionCredentialsProvider(new String[] {ARN, SESSION_NAME});
new KclStsAssumeRoleCredentialsProvider(new String[] {ARN, SESSION_NAME, "endpointRegion=us-east-1"});
}
@Test
public void testAcceptEndpoint() {
// discovered exception during e2e testing; therefore, this test is
// to simply verify the constructed STS client doesn't go *boom*
final KclSTSAssumeRoleSessionCredentialsProvider provider =
new KclSTSAssumeRoleSessionCredentialsProvider(ARN, SESSION_NAME);
final KclStsAssumeRoleCredentialsProvider provider =
new KclStsAssumeRoleCredentialsProvider(ARN, SESSION_NAME, "endpointRegion=us-east-1");
provider.acceptEndpoint("endpoint", "us-east-1");
}
@Test
public void testVarArgs() {
for (final String[] varargs : Arrays.asList(
new String[] {ARN, SESSION_NAME, "externalId=eid", "foo"},
new String[] {ARN, SESSION_NAME, "foo", "externalId=eid"})) {
new String[] {ARN, SESSION_NAME, "externalId=eid", "foo", "endpointRegion=us-east-1"},
new String[] {ARN, SESSION_NAME, "foo", "externalId=eid", "endpointRegion=us-east-1"})) {
final VarArgsSpy provider = new VarArgsSpy(varargs);
assertEquals("eid", provider.externalId);
}
}
private static class VarArgsSpy extends KclSTSAssumeRoleSessionCredentialsProvider {
private static class VarArgsSpy extends KclStsAssumeRoleCredentialsProvider {
private String externalId;

View file

@ -16,16 +16,17 @@ package software.amazon.kinesis.multilang.config;
import java.util.Arrays;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSCredentialsProviderChain;
import com.amazonaws.auth.BasicAWSCredentials;
import lombok.ToString;
import org.hamcrest.Description;
import org.hamcrest.Matcher;
import org.hamcrest.TypeSafeDiagnosingMatcher;
import org.junit.Test;
import software.amazon.kinesis.multilang.auth.KclSTSAssumeRoleSessionCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider;
import software.amazon.kinesis.multilang.auth.KclStsAssumeRoleCredentialsProvider;
import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.CoreMatchers.instanceOf;
@ -33,31 +34,32 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertThat;
public class AWSCredentialsProviderPropertyValueDecoderTest {
public class AwsCredentialsProviderPropertyValueDecoderTest {
private static final String TEST_ACCESS_KEY_ID = "123";
private static final String TEST_SECRET_KEY = "456";
private final String credentialName1 = AlwaysSucceedCredentialsProvider.class.getName();
private final String credentialName2 = ConstructorCredentialsProvider.class.getName();
private final AWSCredentialsProviderPropertyValueDecoder decoder = new AWSCredentialsProviderPropertyValueDecoder();
private final String createCredentialClass = CreateProvider.class.getName();
private final AwsCredentialsProviderPropertyValueDecoder decoder = new AwsCredentialsProviderPropertyValueDecoder();
@ToString
private static class AWSCredentialsMatcher extends TypeSafeDiagnosingMatcher<AWSCredentialsProvider> {
private static class AwsCredentialsMatcher extends TypeSafeDiagnosingMatcher<AwsCredentialsProvider> {
private final Matcher<String> akidMatcher;
private final Matcher<String> secretMatcher;
private final Matcher<Class<?>> classMatcher;
public AWSCredentialsMatcher(String akid, String secret) {
public AwsCredentialsMatcher(String akid, String secret) {
this.akidMatcher = equalTo(akid);
this.secretMatcher = equalTo(secret);
this.classMatcher = instanceOf(AWSCredentialsProviderChain.class);
this.classMatcher = instanceOf(AwsCredentialsProviderChain.class);
}
@Override
protected boolean matchesSafely(AWSCredentialsProvider item, Description mismatchDescription) {
AWSCredentials actual = item.getCredentials();
protected boolean matchesSafely(AwsCredentialsProvider item, Description mismatchDescription) {
AwsCredentials actual = item.resolveCredentials();
boolean matched = true;
if (!classMatcher.matches(item)) {
@ -65,12 +67,12 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
matched = false;
}
if (!akidMatcher.matches(actual.getAWSAccessKeyId())) {
akidMatcher.describeMismatch(actual.getAWSAccessKeyId(), mismatchDescription);
if (!akidMatcher.matches(actual.accessKeyId())) {
akidMatcher.describeMismatch(actual.accessKeyId(), mismatchDescription);
matched = false;
}
if (!secretMatcher.matches(actual.getAWSSecretKey())) {
secretMatcher.describeMismatch(actual.getAWSSecretKey(), mismatchDescription);
if (!secretMatcher.matches(actual.secretAccessKey())) {
secretMatcher.describeMismatch(actual.secretAccessKey(), mismatchDescription);
matched = false;
}
return matched;
@ -84,31 +86,31 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
}
}
private static AWSCredentialsMatcher hasCredentials(String akid, String secret) {
return new AWSCredentialsMatcher(akid, secret);
private static AwsCredentialsMatcher hasCredentials(String akid, String secret) {
return new AwsCredentialsMatcher(akid, secret);
}
@Test
public void testSingleProvider() {
AWSCredentialsProvider provider = decoder.decodeValue(credentialName1);
AwsCredentialsProvider provider = decoder.decodeValue(credentialName1);
assertThat(provider, hasCredentials(TEST_ACCESS_KEY_ID, TEST_SECRET_KEY));
}
@Test
public void testTwoProviders() {
AWSCredentialsProvider provider = decoder.decodeValue(credentialName1 + "," + credentialName1);
AwsCredentialsProvider provider = decoder.decodeValue(credentialName1 + "," + credentialName1);
assertThat(provider, hasCredentials(TEST_ACCESS_KEY_ID, TEST_SECRET_KEY));
}
@Test
public void testProfileProviderWithOneArg() {
AWSCredentialsProvider provider = decoder.decodeValue(credentialName2 + "|arg");
AwsCredentialsProvider provider = decoder.decodeValue(credentialName2 + "|arg");
assertThat(provider, hasCredentials("arg", "blank"));
}
@Test
public void testProfileProviderWithTwoArgs() {
AWSCredentialsProvider provider = decoder.decodeValue(credentialName2 + "|arg1|arg2");
AwsCredentialsProvider provider = decoder.decodeValue(credentialName2 + "|arg1|arg2");
assertThat(provider, hasCredentials("arg1", "arg2"));
}
@ -118,14 +120,34 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
@Test
public void testKclAuthProvider() {
for (final String className : Arrays.asList(
KclSTSAssumeRoleSessionCredentialsProvider.class.getName(), // fully-qualified name
KclSTSAssumeRoleSessionCredentialsProvider.class.getSimpleName() // name-only; needs prefix
)) {
final AWSCredentialsProvider provider = decoder.decodeValue(className + "|arn|sessionName");
KclStsAssumeRoleCredentialsProvider.class.getName(), // fully-qualified name
KclStsAssumeRoleCredentialsProvider.class.getSimpleName(), // name-only; needs prefix
StsAssumeRoleCredentialsProvider.class.getName(), // user passes full sts package path
StsAssumeRoleCredentialsProvider.class.getSimpleName())) {
final AwsCredentialsProvider provider =
decoder.decodeValue(className + "|arn|sessionName|endpointRegion=us-east-1");
assertNotNull(className, provider);
}
}
/**
* Test that OneArgCreateProvider in the SDK v2 can process a create() method
*/
@Test
public void testEmptyCreateProvider() {
AwsCredentialsProvider provider = decoder.decodeValue(createCredentialClass);
assertThat(provider, hasCredentials(TEST_ACCESS_KEY_ID, TEST_SECRET_KEY));
}
/**
* Test that OneArgCreateProvider in the SDK v2 can process a create(arg1) method
*/
@Test
public void testOneArgCreateProvider() {
AwsCredentialsProvider provider = decoder.decodeValue(createCredentialClass + "|testCreateProperty");
assertThat(provider, hasCredentials("testCreateProperty", TEST_SECRET_KEY));
}
/**
* Test that a provider can be instantiated by its varargs constructor.
*/
@ -135,28 +157,25 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
final String className = VarArgCredentialsProvider.class.getName();
final String encodedValue = className + "|" + String.join("|", args);
final AWSCredentialsProvider provider = decoder.decodeValue(encodedValue);
assertEquals(Arrays.toString(args), provider.getCredentials().getAWSAccessKeyId());
final AwsCredentialsProvider provider = decoder.decodeValue(encodedValue);
assertEquals(Arrays.toString(args), provider.resolveCredentials().accessKeyId());
}
/**
* This credentials provider will always succeed
*/
public static class AlwaysSucceedCredentialsProvider implements AWSCredentialsProvider {
public static class AlwaysSucceedCredentialsProvider implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials(TEST_ACCESS_KEY_ID, TEST_SECRET_KEY);
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create(TEST_ACCESS_KEY_ID, TEST_SECRET_KEY);
}
@Override
public void refresh() {}
}
/**
* This credentials provider needs a constructor call to instantiate it
*/
public static class ConstructorCredentialsProvider implements AWSCredentialsProvider {
public static class ConstructorCredentialsProvider implements AwsCredentialsProvider {
private String arg1;
private String arg2;
@ -172,15 +191,12 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
}
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials(arg1, arg2);
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create(arg1, arg2);
}
@Override
public void refresh() {}
}
private static class VarArgCredentialsProvider implements AWSCredentialsProvider {
private static class VarArgCredentialsProvider implements AwsCredentialsProvider {
private final String[] args;
@ -189,13 +205,33 @@ public class AWSCredentialsProviderPropertyValueDecoderTest {
}
@Override
public AWSCredentials getCredentials() {
public AwsCredentials resolveCredentials() {
// KISS solution to surface the constructor args
final String flattenedArgs = Arrays.toString(args);
return new BasicAWSCredentials(flattenedArgs, flattenedArgs);
return AwsBasicCredentials.create(flattenedArgs, flattenedArgs);
}
}
/**
* Credentials provider to test AWS SDK v2 create() methods for providers like ProfileCredentialsProvider
*/
public static class CreateProvider implements AwsCredentialsProvider {
private String accessKeyId;
private CreateProvider(String accessKeyId) {
this.accessKeyId = accessKeyId;
}
public static CreateProvider create() {
return new CreateProvider(TEST_ACCESS_KEY_ID);
}
public static CreateProvider create(String accessKeyId) {
return new CreateProvider(accessKeyId);
}
@Override
public void refresh() {}
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create(accessKeyId, TEST_SECRET_KEY);
}
}
}

View file

@ -22,15 +22,14 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Set;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.google.common.collect.ImmutableSet;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.runners.MockitoJUnitRunner;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.kinesis.common.InitialPositionInStream;
import software.amazon.kinesis.metrics.MetricsLevel;
@ -60,7 +59,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123"
},
'\n'));
@ -77,7 +76,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"applicationName = app",
"streamName = 123",
"AWSCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"AwsCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"workerId = 123",
"failoverTimeMillis = 100",
"shardSyncIntervalMillis = 500"
@ -98,7 +97,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"applicationName = app",
"streamName = 123",
"AWSCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"AwsCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"initialPositionInStreamExtended = " + epochTimeInSeconds
},
'\n'));
@ -116,7 +115,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"applicationName = app",
"streamName = 123",
"AWSCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"AwsCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"initialPositionInStream = AT_TIMESTAMP"
},
'\n'));
@ -136,7 +135,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"applicationName = app",
"streamName = 123",
"AWSCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"AwsCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"initialPositionInStreamExtended = null"
},
'\n'));
@ -151,7 +150,7 @@ public class KinesisClientLibConfiguratorTest {
public void testWithUnsupportedClientConfigurationVariables() {
MultiLangDaemonConfiguration config = getConfiguration(StringUtils.join(
new String[] {
"AWSCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"AwsCredentialsProvider = " + credentialName1 + ", " + credentialName2,
"workerId = id",
"kinesisClientConfig = {}",
"streamName = stream",
@ -170,7 +169,7 @@ public class KinesisClientLibConfiguratorTest {
MultiLangDaemonConfiguration config = getConfiguration(StringUtils.join(
new String[] {
"streamName = kinesis",
"AWSCredentialsProvider = " + credentialName2 + ", " + credentialName1,
"AwsCredentialsProvider = " + credentialName2 + ", " + credentialName1,
"workerId = w123",
"maxRecords = 10",
"metricsMaxQueueSize = 20",
@ -195,7 +194,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD, " + credentialName1,
"AwsCredentialsProvider = ABCD, " + credentialName1,
"workerId = 0",
"cleanupLeasesUponShardCompletion = false",
"validateSequenceNumberBeforeCheckpointing = true"
@ -233,7 +232,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 1",
"metricsEnabledDimensions = ShardId, WorkerIdentifier"
},
@ -253,7 +252,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 123",
"initialPositionInStream = TriM_Horizon"
},
@ -268,7 +267,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 123",
"initialPositionInStream = LateSt"
},
@ -283,7 +282,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 123",
"initialPositionInStream = TriM_Horizon",
"abc = 1"
@ -302,7 +301,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 123",
"initialPositionInStream = TriM_Horizon",
"maxGetRecordsThreadPool = 1"
@ -318,7 +317,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = ABCD," + credentialName1,
"AwsCredentialsProvider = ABCD," + credentialName1,
"workerId = 123",
"initialPositionInStream = TriM_Horizon",
"maxGetRecordsThreadPool = 0",
@ -334,7 +333,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123",
"failoverTimeMillis = 100nf"
},
@ -348,7 +347,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123",
"failoverTimeMillis = -12"
},
@ -380,7 +379,7 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"failoverTimeMillis = 100",
"shardSyncIntervalMillis = 500"
},
@ -397,7 +396,7 @@ public class KinesisClientLibConfiguratorTest {
String test = StringUtils.join(
new String[] {
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123",
"failoverTimeMillis = 100"
},
@ -410,7 +409,7 @@ public class KinesisClientLibConfiguratorTest {
String test = StringUtils.join(
new String[] {
"applicationName = b",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123",
"failoverTimeMillis = 100",
"streamName = ",
@ -425,7 +424,7 @@ public class KinesisClientLibConfiguratorTest {
String test = StringUtils.join(
new String[] {
"streamName = a",
"AWSCredentialsProvider = " + credentialName1,
"AwsCredentialsProvider = " + credentialName1,
"workerId = 123",
"failoverTimeMillis = 100"
},
@ -434,12 +433,12 @@ public class KinesisClientLibConfiguratorTest {
}
@Test
public void testWithAWSCredentialsFailed() {
public void testWithAwsCredentialsFailed() {
String test = StringUtils.join(
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialName2,
"AwsCredentialsProvider = " + credentialName2,
"failoverTimeMillis = 100",
"shardSyncIntervalMillis = 500"
},
@ -457,6 +456,34 @@ public class KinesisClientLibConfiguratorTest {
}
}
@Test
public void testProcessKeyWithExpectedCasing() {
String key = "AwsCredentialsProvider";
String result = configurator.processKey(key);
assertEquals("awsCredentialsProvider", result);
}
@Test
public void testProcessKeyWithOldCasing() {
String key = "AWSCredentialsProvider";
String result = configurator.processKey(key);
assertEquals("awsCredentialsProvider", result);
}
@Test
public void testProcessKeyWithMixedCasing() {
String key = "AwScReDeNtIaLsPrOvIdEr";
String result = configurator.processKey(key);
assertEquals("awsCredentialsProvider", result);
}
@Test
public void testProcessKeyWithSuffix() {
String key = "awscredentialsproviderDynamoDB";
String result = configurator.processKey(key);
assertEquals("awsCredentialsProviderDynamoDB", result);
}
// TODO: fix this test
@Test
public void testWithDifferentAWSCredentialsForDynamoDBAndCloudWatch() {
@ -464,9 +491,9 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialNameKinesis,
"AWSCredentialsProviderDynamoDB = " + credentialNameDynamoDB,
"AWSCredentialsProviderCloudWatch = " + credentialNameCloudWatch,
"AwsCredentialsProvider = " + credentialNameKinesis,
"AwsCredentialsProviderDynamoDB = " + credentialNameDynamoDB,
"AwsCredentialsProviderCloudWatch = " + credentialNameCloudWatch,
"failoverTimeMillis = 100",
"shardSyncIntervalMillis = 500"
},
@ -492,9 +519,9 @@ public class KinesisClientLibConfiguratorTest {
new String[] {
"streamName = a",
"applicationName = b",
"AWSCredentialsProvider = " + credentialNameKinesis,
"AWSCredentialsProviderDynamoDB = " + credentialName2,
"AWSCredentialsProviderCloudWatch = " + credentialName2,
"AwsCredentialsProvider = " + credentialNameKinesis,
"AwsCredentialsProviderDynamoDB = " + credentialName2,
"AwsCredentialsProviderCloudWatch = " + credentialName2,
"failoverTimeMillis = 100",
"shardSyncIntervalMillis = 500"
},
@ -526,71 +553,56 @@ public class KinesisClientLibConfiguratorTest {
/**
* This credentials provider will always succeed
*/
public static class AlwaysSucceedCredentialsProvider implements AWSCredentialsProvider {
public static class AlwaysSucceedCredentialsProvider implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials("a", "b");
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create("a", "b");
}
@Override
public void refresh() {}
}
/**
* This credentials provider will always succeed
*/
public static class AlwaysSucceedCredentialsProviderKinesis implements AWSCredentialsProvider {
public static class AlwaysSucceedCredentialsProviderKinesis implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials("", "");
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create("DUMMY_ACCESS_KEY_ID", "DUMMY_SECRET_ACCESS_KEY");
}
@Override
public void refresh() {}
}
/**
* This credentials provider will always succeed
*/
public static class AlwaysSucceedCredentialsProviderDynamoDB implements AWSCredentialsProvider {
public static class AlwaysSucceedCredentialsProviderDynamoDB implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials("", "");
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create("DUMMY_ACCESS_KEY_ID", "DUMMY_SECRET_ACCESS_KEY");
}
@Override
public void refresh() {}
}
/**
* This credentials provider will always succeed
*/
public static class AlwaysSucceedCredentialsProviderCloudWatch implements AWSCredentialsProvider {
public static class AlwaysSucceedCredentialsProviderCloudWatch implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
return new BasicAWSCredentials("", "");
public AwsCredentials resolveCredentials() {
return AwsBasicCredentials.create("DUMMY_ACCESS_KEY_ID", "DUMMY_SECRET_ACCESS_KEY");
}
@Override
public void refresh() {}
}
/**
* This credentials provider will always fail
*/
public static class AlwaysFailCredentialsProvider implements AWSCredentialsProvider {
public static class AlwaysFailCredentialsProvider implements AwsCredentialsProvider {
@Override
public AWSCredentials getCredentials() {
public AwsCredentials resolveCredentials() {
throw new IllegalArgumentException();
}
@Override
public void refresh() {}
}
private MultiLangDaemonConfiguration getConfiguration(String configString) {

View file

@ -17,10 +17,12 @@ streamName = kclpysample
applicationName = MultiLangTest
# Users can change the credentials provider the KCL will use to retrieve credentials.
# The DefaultAWSCredentialsProviderChain checks several other providers, which is
# Expected key name (case-sensitive):
# AwsCredentialsProvider / AwsCredentialsProviderDynamoDB / AwsCredentialsProviderCloudWatch
# The DefaultCredentialsProvider checks several other providers, which is
# described here:
# http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/DefaultAWSCredentialsProviderChain.html
AWSCredentialsProvider = DefaultAWSCredentialsProviderChain
# https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html
AwsCredentialsProvider = DefaultCredentialsProvider
# Appended to the user agent of the KCL. Does not impact the functionality of the
# KCL in any other way.

View file

@ -23,7 +23,7 @@
<parent>
<groupId>software.amazon.kinesis</groupId>
<artifactId>amazon-kinesis-client-pom</artifactId>
<version>2.6.1-SNAPSHOT</version>
<version>2.7.1-SNAPSHOT</version>
</parent>
<artifactId>amazon-kinesis-client</artifactId>
@ -82,6 +82,12 @@
<groupId>software.amazon.glue</groupId>
<artifactId>schema-registry-serde</artifactId>
<version>${gsr.version}</version>
<exclusions>
<exclusion>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-sts</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>software.amazon.glue</groupId>

View file

@ -49,7 +49,7 @@ public class RetrievalConfig {
*/
public static final String KINESIS_CLIENT_LIB_USER_AGENT = "amazon-kinesis-client-library-java";
public static final String KINESIS_CLIENT_LIB_USER_AGENT_VERSION = "2.6.1-SNAPSHOT";
public static final String KINESIS_CLIENT_LIB_USER_AGENT_VERSION = "2.7.1-SNAPSHOT";
/**
* Client used to make calls to Kinesis for records retrieval

View file

@ -6,26 +6,42 @@ KCL multilang does not, and is not intended to, proxy the full breadth of the AW
However, KCL now provides better extensibility to handle, and be enhanced to handle, additional configurations.
This document should help multilang customers configure a suitable `CredentialProvider` (or contribute changes to support a new use case!).
In KCL versions prior to 2.7, the `AWSCredentialsProvider` property used AWS SDK for Java 1.x provider names.
When upgrading to KCL 2.7 or later, you must update to use AWS SDK v2 credentials provider names in your configuration.
For a complete mapping of credential provider names between version 1.x and 2.x, see the [AWS SDK for Java migration guide — Credentials provider changes mapped between versions 1.x and 2.x](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/migration-client-credentials.html#credentials-changes-mapping).
You also can refer to the following examples.
## Sample Provider Configuration
In a Properties file, an `AWSCredentialsProperty` configuration might look like:
In a Properties file, an `AwsCredentialsProperty` configuration might look like:
```
AWSCredentialsProvider = STSAssumeRoleSessionCredentialsProvider|<arn>|<sessionName>
AwsCredentialsProvider = StsAssumeRoleCredentialsProvider|<arn>|<sessionName>|endpointRegion=us-east-1
```
This basic configuration creates an [STSAssumeRoleSessionCredentialsProvider][sts-assume-provider] with an ARN and session name.
This basic configuration creates an [StsAssumeRoleCredentialsProvider][sts-assume-provider] with an ARN and session name.
If `endpointRegion` is not specified, the provider will use the region defined in the AWS config file.
If no region is found in the config file, the credentials provider will fail.
The providers generated by this config property will be [AWS SDK v2 AwsCredentialsProviders][aws-credentials-provider].
These differ from the SDK v1 AWSCredentialsProviders in a number of ways. See [Credentials Provider Changes][credentials-provider-changes].
While functional, this configuration is limited.
For example, this configuration cannot set a regional endpoint (e.g., VPC use case).
Leveraging nested properties, an `AWSCredentialsProperty` value might change to:
Leveraging nested properties, an `AwsCredentialsProperty` value might change to:
```
AWSCredentialsProvider = KclSTSAssumeRoleSessionCredentialsProvider|<arn>|<sessionName>\
AwsCredentialsProvider = KclStsAssumeRoleCredentialsProvider|<arn>|<sessionName>\
|endpointRegion=us-east-1|externalId=spartacus
```
N.B. Backslash (`\`) is for multi-line legibility and is not required.
You can create a default [DefaultCredentialsProvider][default-credentials-provider] by passing it in the config like:
```
AwsCredentialsProvider = DefaultCredentialsProvider
```
## Nested Properties
KCL multilang supports "nested properties" on the `AWSCredentialsProvider` key in the properties file.
KCL multilang supports "nested properties" on the `AwsCredentialsProvider` key in the properties file.
The [Backus-Naur form][bnf] of the value:
```
<property-value> ::= <provider-class> ["|" <required-param>]* ["|" <nested-property>]*
@ -36,8 +52,9 @@ The [Backus-Naur form][bnf] of the value:
<nested-value ::= <string> # this depends on the nested key
```
In general, required parameters are passed directly to the class' constructor
(e.g., [STSAssumeRoleSessionCredentialsProvider(String, String)][sts-assume-provider-constructor]).
In general, required parameters are passed directly to the class' constructor or .create() method
(e.g., [ProfileCredentialsProvider(String)][profile-credentials-provider-create]). However, most of these providers
require builders and will require a custom implementation similar to `KclStsAssumeRoleCredentialsProvider` for customization
Nested properties are a custom mapping provided by KCL multilang, and do not exist in the AWS SDK.
See [NestedPropertyKey][nested-property-key] for the supported keys, and details on their expected values.
@ -54,18 +71,27 @@ A backwards-compatible addition might look like:
}
```
### KclSTSAssumeRoleSessionCredentialsProvider
KCL multilang includes a [custom nested property processor for `STSAssumeRole`][kcl-sts-provider].
Multilang configurations that use `STSAssumeRoleSessionCredentialsProvider` need only prefix `Kcl` to exercise this new provider:
Leveraging nested properties, an `AwsCredentialsProperty` value might look like:
```
AWSCredentialsProvider = KclSTSAssumeRoleSessionCredentialsProvider|<arn>|<sessionName>
AwsCredentialsProvider = KclStsAssumeRoleCredentialsProvider|<arn>|<sessionName>\
|endpointRegion=us-east-1|externalId=spartacus
```
N.B. Backslash (`\`) is for multi-line legibility and is not required.
### KclStsAssumeRoleCredentialsProvider
KCL multilang includes a [custom nested property processor for `StsAssumeRole`][kcl-sts-provider].
Multilang configurations that use `StsAssumeRoleSessionCredentialsProvider` need only prefix `Kcl` to exercise this new provider:
```
AwsCredentialsProvider = KclStsAssumeRoleCredentialsProvider|<arn>|<sessionName>|endpointRegion=us-east-1
```
[aws-credentials-provider]: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/AwsCredentialsProvider.html
[bnf]: https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form
[kcl-sts-provider]: /amazon-kinesis-client-multilang/src/main/java/software/amazon/kinesis/multilang/auth/KclSTSAssumeRoleSessionCredentialsProvider.java
[kcl-sts-provider]: /amazon-kinesis-client-multilang/src/main/java/software/amazon/kinesis/multilang/auth/KclStsAssumeRoleCredentialsProvider.java
[nested-property-key]: /amazon-kinesis-client-multilang/src/main/java/software/amazon/kinesis/multilang/NestedPropertyKey.java
[nested-property-processor]: /amazon-kinesis-client-multilang/src/main/java/software/amazon/kinesis/multilang/NestedPropertyProcessor.java
[sts-assume-provider]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleSessionCredentialsProvider.html
[sts-assume-provider-constructor]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/STSAssumeRoleSessionCredentialsProvider.html#STSAssumeRoleSessionCredentialsProvider-java.lang.String-java.lang.String-
[sts-assume-provider]: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/sts/auth/StsAssumeRoleCredentialsProvider.html
[profile-credentials-provider-create]: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/ProfileCredentialsProvider.html#create(java.lang.String)
[default-credentials-provider]: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html
[credentials-provider-changes]: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/migration-client-credentials.html

View file

@ -22,7 +22,7 @@
<artifactId>amazon-kinesis-client-pom</artifactId>
<packaging>pom</packaging>
<name>Amazon Kinesis Client Library</name>
<version>2.6.1-SNAPSHOT</version>
<version>2.7.1-SNAPSHOT</version>
<description>The Amazon Kinesis Client Library for Java enables Java developers to easily consume and process data
from Amazon Kinesis.
</description>