Datasync between s3 buckets failing "Ensure bucket access role has s3:ListBucket permission

Hi, i ran into the same issue. i have solved it for myself. on the command line you will have to assume that role before you can run this command successfully.

My role for datasync in account A is OPS-173

you will have to do 2 things - a. Your role trust relationship (in OPS-173) will have to be modified. so that your default cli role can assume your new role. below is my example

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "datasync.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        },
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:sts::7060434:assumed-role/Admin/vya@sabc.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

b. once you add the above, then on cli you have to temporarily assume this role. for that

##### INITIAL ROLE
Downloads % aws sts get-caller-identity      
{
    "UserId": "JHNQ4T5TK2A:vya@singlestore.com",
    "Account": "706058434",
    "Arn": "arn:aws:sts::70608434:assumed-role/Admin/vya@sabc.com"
}
Downloads % 

########## ASSUMING ROLE
Downloads % aws sts assume-role --role-arn "arn:aws:iam::70604434:role/OPS-173" --role-session-name AWSCLI-Session

{
    "Credentials": {
        "AccessKeyId": "MRLPY",
        "SecretAccessKey": "nCwklbz",
        "SessionToken": "QAxo9osIBQ==",
        "Expiration": "2023-06-30T12:19:37+00:00"
    },
    "AssumedRoleUser": {
        "AssumedRoleId": "AROARHI56DUK4J:AWSCLI-Session",
        "Arn": "arn:aws:sts::70658434:assumed-role/OPS-173/AWSCLI-Session"
    }
}

###### EXPORTING NEW ROLE CREDENTIALS IN ENV 
Downloads % export AWS_ACCESS_KEY_ID=MRLPY
Downloads % export AWS_SECRET_ACCESS_KEY=nCwkLbz
Downloads % export AWS_SESSION_TOKEN=QAxo9osIBQ==
Downloads % 


###### VERIFYING THAT NEW CREDENTIALS ARE EXPORTED AND ROLE ASSUMED
Downloads % aws sts get-caller-identity
{
    "UserId": "AROA2IY2I56DUK4J:AWSCLI-Session",
    "Account": "706434",
    "Arn": "arn:aws:sts::70608434:assumed-role/OPS-173/AWSCLI-Session"
}

then you will be able to run this command successfully

Downloads % aws datasync create-location-s3 \
  --s3-bucket-arn arn:aws:s3:::backup-employees1 \
  --s3-config '{"BucketAccessRoleArn":"arn:aws:iam::706434:role/OPS-173"}'                 
{
    "LocationArn": "arn:aws:datasync:us-east-1:7068434:location/loc-06d1bfc535"
}
Downloads %

Hope this helps !

Hi, Please confirm that IAM role which configured with CLI in account A i.e, 'name-of-your-role' must have identity-based policy which has permission(s3:ListBucket) to access Account B bucket. IAM evaluates both identity & resource based policies to allow/deny request for cross-account access. The same 'name-of-datasync-role' also needs permissions in both places. If this is already present, it should work fine. Most likely your cli IAM role 'name-of-your-role' does not have ListBucket permission in Account A. https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic-cross-account.html

I had to run the CLI command while logged in as a user that had assumed the role created in Step 1 of the tutorial. AWS also mentions this at https://docs.aws.amazon.com/datasync/latest/userguide/troubleshooting-datasync-locations-tasks.html#troubleshoot-cross-account-s3.

See https://stackoverflow.com/a/68825289/1519256 for more details.

If you see this error when trying to create your DataSync destination location for a cross-account transfer ...

An error occurred (InvalidRequestException) when calling the CreateLocationS3 operation: DataSync location access test failed: could not perform s3:HeadBucket on bucket MY-BUCKET. Access denied. Ensure bucket access role has s3:ListBucket permission.

... it typically means that your source account's user permissions don't include the s3:ListBucket permission. To resolve this, add s3:ListBucket to your user permissions and try creating the destination location again.

Tip: Also, since the cross-account transfer flow asks you to jump around the console and CLI while assuming different roles, I find it much more convenient to only switch roles in the console. I can open CloudShell in the console if I need to run any CLI commands, eliminating the need to temporarily assume a role in my terminal.

I resort to using ARN of the user instead of the role in the bucket policy. It works so it means the problem seems to be with the role apparently. If you don't want to wrestle with the role stuff for now so you can try this.

        {
            "Sid": "DataSyncCreateS3Location",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::USER_ACCOUNT_ID:user/USER_NAME"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::DESTINATION_BUCKET"
        }

Greetings

This error message indicates that there is an issue with the permissions of the AWS DataSync service when trying to access your S3 bucket. To troubleshoot this issue, follow these steps:

Double-check the IAM role:

Ensure that you have created an IAM role with the correct permissions as described in the tutorial. Verify that the IAM role has a trust relationship with the DataSync service. Review the IAM role policy:

Verify that the IAM policy attached to the role has the required permissions, specifically s3:ListBucket and s3:HeadBucket for the specified bucket. Check if the policy has the correct "Resource" specified, which should be the ARN of your S3 bucket. Ensure there are no typos or missing characters in the ARN. Here's an example of a policy that grants the necessary permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:HeadBucket"
      ],
      "Resource": "arn:aws:s3:::name-of-my-bucket"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::name-of-my-bucket/*"
    }
  ]
}

Verify bucket ownership and access:

Confirm that you have ownership of the S3 bucket and that there are no bucket policies or Access Control Lists (ACLs) restricting access to the bucket. Check the bucket name:

Make sure that the bucket name you're using in the CreateLocationS3 API call is correct and that it exists in your account. If you've verified all these steps and are still facing issues, try creating a new IAM role and policy with the required permissions, and use that new role in the CreateLocationS3 API call. Please let me know if I answered your question