Error while configuring Athena data source on Grafana

Question

I am trying to configure Athena as a data source on Grafana but I am getting the below error. It's in the same region, Grafana role has all permissions to Athena, S3 and Glue. Grafana role's trust policy has STS: Assume role as Admin role.

Grafana Role's Trust Policy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ADMIN-role"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

ERROR:
operation error Athena: ListWorkGroups, get identity: get credentials: failed to refresh cached credentials, operation error STS: AssumeRole, https response error StatusCode: 403, Req:123, api error InvalidClientTokenId: The security token included in the request is invalid.Trace:123

re:Post Agent · Answer

The error you're encountering when configuring Athena as a data source in AWS Managed Grafana appears to be related to IAM permissions and authentication issues. Let me help you troubleshoot this.

The error message indicates an issue with the STS AssumeRole operation, specifically that the security token is invalid. This suggests there's a problem with how your Grafana workspace is trying to assume the role you've configured.

Here are some steps to resolve this issue:

1. **Check your authentication provider configuration**: When setting up the Athena data source in Grafana, make sure you're using "Workspace IAM Role" as the authentication provider in the Connection Details menu. This is the recommended approach.

2. **Verify service-managed permissions**: Ensure your Grafana workspace is using service-managed permissions rather than customer-managed permissions. To do this:
- Go to the Amazon Managed Grafana console
- Select your workspace
- Check if it's using service-managed permissions
- If not, edit the IAM role settings and switch to "Service managed" and save changes

3. **Enable the Athena data source properly**: 
- In the Amazon Managed Grafana console, go to the Data sources tab
- Select the checkbox for Amazon Athena
- Choose Actions, then "Enable service-managed policy"

4. **Review your trust policy**: The trust policy you've shared shows that your Grafana role trusts an "ADMIN-role" to assume it, but for Grafana to access Athena, the trust relationship should be the other way around - the Grafana service should be able to assume the role with Athena permissions.

5. **Check region consistency**: Confirm that your Athena workgroup and Grafana workspace are in the same AWS region.

By following these steps, particularly ensuring that you're using service-managed permissions and properly enabling the Athena data source through the Amazon Managed Grafana console, you should be able to resolve the authentication issue.  
**Sources**  
[Use AWS data source configuration to add Amazon Athena as a data source - Amazon Managed Grafana](https://docs.aws.amazon.com/grafana/latest/userguide/Athena-adding-AWS-config.html)  
[Using Athena data source - Amazon Managed Grafana](https://docs.aws.amazon.com/grafana/latest/userguide/Athena-using-the-data-source.html)  
[Manually adding the Athena data source - Amazon Managed Grafana](https://docs.aws.amazon.com/grafana/latest/userguide/Athena-add-the-data-source.html)  
[AWS Managed Grafana workspace cannot be configured with Athena workgroup | AWS re:Post](https://repost.aws/questions/QUfoy6ht0eQCOM9IzRHO7g8g/aws-managed-grafana-workspace-cannot-be-configured-with-athena-workgroup)

Error while configuring Athena data source on Grafana

Add your answer

Relevant content