security token included in the request is invalid on EC2 Instance with IAM role.

Question

Hello, I am working on a connector that uses the Amazon Connect chat APIs. The connecter is working fine on my local host machine. Now I deploy it on the EC2 instance after creating a Docker image using ECR and setting up the Docker container.

The Amazon Connect Chat API uses AWS credentials for this purpose. I attached the IAM role to an EC2 instance with full connection access. and change the code accordingly to fetch the credentials from the IAM role, but it's not working. When I hard code the credentials on the EC2 instance, it works fine, but when I remove them and change the code for using the environment, it stops working and throws the error "Error: The security token included in the request is invalid." . I don't know what the issue is; AWS cli is installed on the EC2 instance.

I am using the AWS4 module for creating the headers in my code.

```
import aws4 from 'aws4';
 /*
  const credentials = {
      
    accessKeyId:  process.env.aws_access_key_id,
    secretAccessKey: process.env.aws_secret_access_key,
     sessionToken: process.env.aws_session_token
  }; */
  const opts = {
       
      host: 'connect.us-east-1.amazonaws.com',
       method: 'PUT',
      path: '/contact/chat',
      body: JSON.stringify(body),
      service: 'connect',
       };
   
      //aws4.sign(opts,credentials);
       aws4.sign(opts);
```

this is my credentials part on EC2.

Answer

Hello,

Thank you for querying in this forum.

From the descriptions, it seems like although you’ve attached an IAM role with full access and valid credentials to your EC2 instance, you are facing the below error when trying to fetch these credentials from IAM role.

`Error: The security token included in the request is invalid. 
`

This error is typically related to the AWS credentials and often happens when the access key/secret key is incorrect. However, as it works when you hard code/provide credentials manually, we can conclude that the IAM credentials are not malformed.

In most cases, customer setup credentials in a file called credentials found in ~/.aws/ directory that is used when checking credentials and preferred over IAM role attached to EC2 instance. There is a great chance that you have a stale credentials (either user credentials deleted or inactive) in your ~/.aws/credentials file that needs to be remove or change the word "default" to something else in the file.

At times, there is also an aws_session_token in the [default] profile of the credentials file that was probably left over in the credentials file from a previous use, and `$ aws configure` overwrote the access key and secret key, but did not delete the old session token [1].

In other words, it is fetching credentials different from the ones you expect it to use, such as from a credentials file instead of an instance profile. This is because credentials file has a higher precedence than the instance profile so they would be used first [2].

I suggest you to check the following :

1. Make sure that AWS CLI is properly installed and configured over the instance [3]. Check $ aws configure command and input the necessary information if not already done,

2. If already done, check if any credentials were configured using the "aws configure" command on the instance earlier. For checking credential configuration, use the command $ aws configure list.

3.  Please look at ~/.aws/credentials and ~/.aws/config as well

4. Backup the credentials file to a new name and then delete the credentials file. Then run “$ aws configure list" again and see if the CLI is now correctly using the instance profile.

However, if you still continue to face this issue, we would be required to perform resource level troubleshooting to investigate further. Hence please create a support case with AWS Premium Support and we will be glad to assist you.

Thank you for your interest in re:Post community. Have a great day!

References
=========
1. https://stackoverflow.com/questions/34582318/how-can-i-resolve-the-error-the-security-token-included-in-the-request-is-inval
2. https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#configure-precedence
3. https://aws.amazon.com/cli/

security token included in the request is invalid on EC2 Instance with IAM role.

References

Relevant content