How can I troubleshoot access denied or unauthorized operation errors with an IAM policy?

9 minute read
0

I am trying to perform an action on an AWS resource and I received an "access denied" or "unauthorized operation" error. How can I troubleshoot these permission issues?

Short description

To troubleshoot issues with AWS Identity and Access Management (IAM) policies:

Resolution

Identify the API caller and understand the error messages

Important:

Be sure that the API calls are made on behalf of the correct IAM entity before reviewing IAM policies. If the error message doesn't include the caller information, then follow these steps to identify the API caller:

  1. Open the AWS Management Console.
  2. In the upper-right corner of the page, choose the arrow next to the account information.
  3. If you're signed in as an IAM role, refer to "Currently active as" for the assumed role's name, and "Account ID" for account ID.
  4. If you're signed in as a federated user, refer to "Federated User" for the federation role name and role session name.

-or-

Use the AWS CLI command get-caller-identity to identify the API caller. You can also use the AWS CLI command with the --debug flag to identify the source of the credentials from the output similar to the following:

2018-03-13 16:23:57,570 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials

Check the IAM policy permissions

Verify if the necessary permissions are granted to the API caller by checking the attached IAM policies. For more information, see Determining whether a request is allowed or denied within an account.

Evaluate AWS Organizations SCPs

If the AWS account is a part of an AWS Organization, SCPs can be applied at the hierarchical level to allow or deny actions. The SCP permissions are inherited by all IAM entities in the AWS account. Make sure that the API caller isn't explicitly denied in the SCP.

Make sure that the API being called isn't explicitly denied in an Organizational SCP policy that impacts the caller

Review identity-based and resource-based policies

Make sure that there is an explicit allow statement in the IAM entities identity-based policy for the API caller. Then, make sure that the API supports resource-level permissions. If the API caller doesn't support resource-level permissions, make sure the wildcard "*" is specified in the resource element of the IAM policy statement.

You can attach resource-based policies to a resource within the AWS service to provide access. For more information, see Identity-based policies and resource-based policies.

To view the IAM policy summary:

  1. Open the IAM console.
  2. In the navigation pane, choose Policies.
  3. Choose the arrow next to the policy name to expand the policy details view.

In the following example, the policy doesn't work because not all Amazon Elastic Compute Cloud (Amazon EC2) API actions support resource-level permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "SorryThisIsNotGoingToWorkAsExpected",
            "Effect": "Allow",
            "Action": ["ec2:*"],
            "Resource": "arn:aws:ec2:us-east-1:accountid:instance/*"
        }
    ]
}

IAM users that try to launch an Amazon EC2 instance in the us-east-1 Region with the run-instances AWS CLI command receive an error message similar to the following:

"An error occurred (UnauthorizedOperation) when calling the RunInstances operation: You are not authorized to perform this operation. Encoded authorization failure message:…"

To resolve this, change the resource to a wildcard "*". This is because Amazon EC2 only supports partial resource-level permissions.

To decode the authorization failure message to get more details on the reason for this failure, use the DecodeAuthorizationMessage API action similar to the following:

$ aws sts decode-authorization-message --encoded-message <encoded-message-from-the-error>

Check for permission boundaries

If the IAM entity has a permission boundary attached, the boundary sets the maximum permissions that the entity has.

Evaluate session policies

If the API caller is an IAM role or federated user, session policies are passed for the duration of the session. The permissions for a session are the intersection of the identity-based policies for the IAM entity used to create the session and the session policies. Make sure that the API call exists in the IAM policy and entity.

Make sure that the condition keys in the policy are supported by the APIs

AWS condition keys can be used to compare elements in an API request made to AWS with key values specified in a IAM policy. The condition keys can either be a global condition key or defined by the AWS service. AWS service specific condition keys can only be used within that service (for example EC2 conditions on EC2 API actions).For more information, see Actions, resources, and condition context keys for AWS services.

A condition element can contain multiple conditions, and within each condition block can contain multiple key-value pairs. For more information, see Creating a condition with multiple keys or values.

In this example policy, the condition element is matched if an IAM API request is called by the IAM user admin and the source IP address is from 1.1.1.0/24 or 2.2.2.0/24.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "aws:username": "admin"
        },
        "IpAddress": {
          "aws:SourceIp": [
            "1.1.1.0/24",
            "2.2.2.0/24"
          ]
        }
      }
    }
  ]
}

IAM policy errors and troubleshooting examples

See the following examples to identify the error message, the API caller, the API, and the resources being called:

Example error messageAPI callerAPIResourcesWhen
A: "An error occurred (UnauthorizedOperation) when calling the DescribeInstances operation: You are not authorized to perform this operation."unknownDescribeInstancesunknowntime of the error occurs
B: "An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::123456789012:user/test is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456789012:role/EC2-FullAccess"arn:aws:iam::123456789012:user/testAssumeRolearn:aws:iam::123456789012:role/EC2-FullAccesstime of the error occurs
C: "An error occurred (AccessDenied) when calling the GetSessionToken operation: Cannot call GetSessionToken with session credentials"unknownGetSessionTokenunknowntime of the error occurs
D: "An error occurred (UnauthorizedOperation) when calling the AssociateIamInstanceProfile operation: You are not authorized to perform this operation. Encoded authorization failure message: ...."unknownAssociateIamInstanceProfileunknowntime of the error occurs

Using this evaluation method, you can identify the cause of the error messages you can receive for permission issues for different AWS services. For more details, see the following error messages and troubleshooting steps:

Example error message A:

This error message indicates that you don't have permission to call the DescribeInstances API.

To resolve this error, follow these steps:

  1. Identify the API caller.
  2. Confirm that the ec2:DescribeInstances API action isn't included in any deny statements.
  3. Confirm that the ec2:DescribeInstances API action is included in the allow statements.
  4. Confirm that there's no resource specified for this API action. Note: This API action doesn't support resource-level permissions.
  5. Confirm that all IAM conditions specified in the allow statement are supported by the DescribeInstances action and that the conditions are matched.

For more information, see DescribeInstanceStatus.

Example error message B:

This error message includes the API name, API caller, and target resource. Be sure that the IAM identity that called the API has the correct access to the resources. Review the IAM policies using the previous evaluation method.

To resolve this error, follow these steps to confirm the trust policy of IAM role: EC2-FullAccess:

  1. Confirm arn:aws:iam::123456789012:user/test or arn:aws:iam::123456789012:root isn't included in any deny statement of the trust policy.
  2. Confirm arn:aws:iam::123456789012:user/test or arn:aws:iam::123456789012:root is included in the allow statement of the trust policy.
  3. Confirm all IAM conditions specified in that allow statement are supported by sts:AssumeRole API action and matched.

Follow these steps to confirm the IAM policies attached to the API caller (arn:aws:iam::123456789012:user/test):

  1. Confirm arn:aws:iam::123456789012:role/EC2-FullAccess isn't included in any deny statement with sts:AssumeRole API action.
  2. If arn:aws:iam::123456789012:root is in the allow statement of the trust policy, then confirm arn:aws:iam::123456789012:role/EC2-FullAccess is included in the allow statement of the IAM policies with sts:AssumeRole API action.
  3. Confirm all IAM conditions specified in that allow statement are supported by sts:AssumeRole API action and match.

Example error message C:

This error message indicates that get-session-token isn't supported by temporary credentials. For more information, see Comparing the AWS STS API operations.

Example error message D:

This error message returns an encoded message that can provide details about the authorization failure. To decode the error message and get the details of the permission failure, see DecodeAuthorizationMessage. After decoding the error message, identify the API caller and review the resource-level permissions and conditions.

To resolve this error, follow these steps to review the IAM policy permissions:

  1. If the error message indicates that the API is explicitly denied, then remove ec2:AssociateIamInstanceProfile or iam:PassRole API actions from the matched statement.
  2. Confirm that ec2:AssociateIamInstanceProfile and iam:PassRole are in the allow statement with supported and correct resource targets. For example, confirm that the resource targets of ec2:AssociateIamInstanceProfile API action are EC2 instances and the resource targets of iam:PassRole are IAM roles.
  3. If ec2:AssociateIamInstanceProfile and iam:PassRole API actions are in the same allow statement, confirm that all conditions are supported by ec2:AssociateIamInstanceProfile and iam:PassRole API action and that the conditions match.
  4. If ec2:AssociateIamInstanceProfile and iam:PassRole API actions are in separate allow statements, confirm that all conditions in each allow statement are supported by an action and that the conditions match.

For more information, see Policy evaluation logic and Determining whether a request is allowed or denied within an account.


Related information

Troubleshoot IAM policies

Why did I receive an "AccessDenied" or "Invalid information" error trying to assume a cross-account IAM role?

IAM JSON policy elements reference