I have a service running in a namespace that has been using Kube2IAM in order to assume a role that gives it access to S3.
I am trying to remove Kube2IAM and replace it with IRSA to assume the same role through a service account.
I have an OIDC provider and I have updated the role as per the documentation:
ARN: arn:aws:iam::{{account-id}}:role/{{namespace}}-{{serviceName}}
Permission Policy: AmazonS3FullAccess
Trust Relationship:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::{{account-id}}:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/{{id}}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/{{id}}:sub": "system:serviceaccount:{{namespace}}:{{serviceName}},
"oidc.eks.us-east-1.amazonaws.com/id/{{id}}:aud": "sts.amazonaws.com"
}
}
}
]
}
I have then updated the configuration for my service in kuberneties to use the service account:
serviceAccount:
enabled: true
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::{{account-id}}:role/{{namespace}}-{{serviceName}}
However when my service tries to upload to S3 I get:
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: JXJP50TFCCP7J90S; S3 Extended Request ID: H2SpLJ5yEoPcpee91y5mNyEozIChJ8cGZYnoRiabW7Bkzn7/FZDqxolsPnFCeE0mv6dvpgn+E6g=)
The service itself is using the AWS JDK SDK v1 and I have confirmed that it is running at least the version 1.11.704 (as per: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html).
I have attempted to query CloudTrail to find additional information about why the request is resulting in a 403 but no query has yielded any results, not even when running the service using the kube2IAM method I was previously using. so at this point I cannot tell if the service is actually assuming the role or if there is some other issue.
Is there anything I might have missed in order to get this to work?
AWS_DEFAULT_REGION : us-east-1 AWS_REGION : us-east-1 AWS_ROLE_ARN : arn:aws:iam::{{account-id}}:role/{{namespace}}-{{serviceName}} AWS_WEB_IDENTITY_TOKEN_FILE : /var/run/secrets/eks.amazonaws.com/serviceaccount/token
I can also see the service when I list the service accounts using the
kubectl get serviceaccount
command so it does have a service account