credentials working with CLI but not with Java SDK
I'm having trouble getting a set of credentials to work with the Java SDK, when they work with the CLI.
Background - I had some code working on an AWS Elastic Beanstalk instance, where I was setting the environment variables "aws.accessKeyId" and "aws.secretKey", and using the SystemPropertiesCredentialsProvider to build clients for accessing SQS, S3, etc. Following a security review by an internal team, I'm attempting to update this to use a different method of finding credentials - namely, storing the credentials in an external file, instead of an environment variable. To that end, here's what I've done:
I'm using an IAM user on this account, which belongs to a group that has the AmazonSQSFullAccess policy (among others) attached. This is unchanged from the working version of my app, where I had the same user but just a different credentials provider.
I have regenerated the security credentials for this user and verified it is active.
To test, I have the following set up locally in my ~/.aws/credentials file:
[sdk_temp_test] aws_access_key_id = <redacted> aws_secret_access_key = <redacted>
and ~/.aws/config file:
[profile sdk_temp_test] region = us-east-1
- At a shell prompt, if I then do "export AWS_PROFILE=sdk_temp_test" I can run the following commands that show that the credentials work and are able to access basic SQS functionality - I'm not including the output here, but the returned data from the following commands shows that I am calling CLI functions as the user I expect, and I am retrieving the queues I expect to see in the us-east-1 region for this account.
aws sts get-caller-identity aws sqs list-queues
So far, so good. However, I then attempt to do something like the following:
- create a file called "localtest.properties" that contains the following and is accessible on the classpath of my Java application:
- run code like so, this is in a standalone example that illustrates the problem:
AWSCredentialsProvider provider = new ClasspathPropertiesFileCredentialsProvider("localtest.proper ties"); AWSCredentials credentials = provider.getCredentials(); String accessKeyId = credentials.getAWSAccessKeyId(); String secret = credentials.getAWSSecretKey(); System.out.println("accesskey is '" + accessKeyId + "'; secret is '" + secret + "'"); AmazonSQSClient client = (AmazonSQSClient)AmazonSQSClientBuilder.standard() .withRegion(Regions.US_EAST_1) .withCredentials(provider) .build(); System.out.println("LIST QUEUES TEST"); ListQueuesResult lqr = client.listQueues();
This debug line correctly prints out the credentials I expect. But when attempting to listQueues it throws the following exception:
Exception in thread "main" com.amazonaws.services.sqs.model.AmazonSQSException: The security token included in the request is invalid. (Service: AmazonSQS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: <redacted>; Proxy: null)
So I'm a little stuck. The credentials are good, b/c they work on my CLI test. My code I think is ok; I am just switching my credentials provider. And the new provider appears to be finding the correct credentials based on the output debugging. But - put it all together, and it is not working for me when trying an SDK call, as I get that exception. How do I troubleshoot this? Is it possible to get more details beyond the "InvalidClientTokenId" - what specifically is wrong? Can I look up the request ID somewhere to troubleshoot? Does the ClasspathPropertiesFileCredentialsProvider need something that the SystemPropertiesCredentialsProvider I used to use did not?
I opened a ticket with AWS support and they said SDK issues were a little out of scope; they pointed me towards articles on the credentials chain, and some sample code for the NodeJS SDK which is structured a little differently. re: the credentials chain, I think with a custom provider I should bypass that? Just in case, I've ensured there are no environment variables like AWS_ACCESS_KEY_ID, no Java properties like aws.accessKeyId, I've even temporarily deleted my ~/.aws/credentials and config files while running the above code, to make sure that no other credential is "sneaking in", but I still get the same exception.
I do get some warnings while running the above Java code:
Feb 21, 2022 12:00:45 PM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code. (this repeats a number of times) WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.amazonaws.util.XpathUtils (file:/Users/tfeiler/.m2/repository/com/amazonaws/aws-java-sdk-core/1.11.964/aws-java-sdk-core-1.11.964.jar) to method com.sun.org.apache.xpath.internal.XPathContext.getDTMManager() WARNING: Please consider reporting this to the maintainers of com.amazonaws.util.XpathUtils WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release ( this occurs right before the exception is thrown)
They are just warnings, and I think the profile prefix warning is related to the entry in my ~/.aws/config file; so I don't think this is related to my problem, but including it here just in case.
Anyone got any advice on things to try or how to troubleshoot this?
Never did get this working; eventual solution was to use DefaultAWSCredentialsProviderChain, and then add the appropriate policies to the instance profile role that my Elastic Beanstalk instance was running as. When running in a local dev environment I could still use environment variables (or an active CLI profile) and the DefaultAWSCredentialsProviderChain would pick it up either way. I'd prefer to know why I couldn't get the classpath file approach working, but it's a decent workaround.
"The security token included in the request is invalid." normally indicates that the Access Key and Secret you are passing to the Java SDK are invalid or not correctly formatted. When you print the values out, are you able to compare them to what you have stored in the work .aws/config file? E.g. No line breaks or extra characters included in the response.
We do have documentation for both versions of the Java SDK available here on how to use credentials: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html
Additionally we also have the GitHub for the Java SDK which is another good place to post this question: https://github.com/aws/aws-sdk-java/issues
Use KMS grant to access to encrypted KMS - CMK S3 bucketasked 2 months ago
AWS CLI commands after AWS SDK credential store authenticationAccepted Answerasked 3 months ago
When I try to update doc ver I get a code 404asked 2 years ago
credentials working with CLI but not with Java SDKasked 3 months ago
Cognito Java examplesAccepted Answerasked 2 months ago
GameLift Realtime SDK protobuf definitionsasked 5 months ago
How to send messages with GameLift Client in Javaasked 5 months ago
Java IoT connection kit won't build...Accepted Answerasked 24 days ago
Getting started with MQTT client in Java SDK?
Simulate a "thing" in Java?