My Amazon S3 bucket has data files created by the UNLOAD command from another account. Why can't I access those files?

4 minute read
0

My Amazon Simple Storage Service (Amazon S3) bucket has data files that were created by the UNLOAD command from another account's Amazon Redshift cluster. When I try to access those files from my account, I get 403 Access Denied errors.

Short Description

When another account creates Amazon Redshift data files from the UNLOAD command and transfers them to your bucket, you gain ownership of those files. This is because Amazon S3 sets Object Ownership to bucket owner enforced by default. This action also turns off access control lists (ACLs).

However, if your use case requires that you turn on ACLs, then you can't access data files that transfer to your bucket from another account.

To get access to the data files, an AWS Identity and Access Management (IAM) role with cross-account permissions must run the UNLOAD command again. Follow these steps to set up the Amazon Redshift cluster with cross-account permissions to the bucket:

  1. From the account of the S3 bucket, create an IAM role (bucket role) with permissions to the bucket.
  2. From the account of the Amazon Redshift cluster, create another IAM role (cluster role) with permissions to assume the bucket role.
  3. Update the bucket role to grant bucket access and create a trust relationship with the cluster role.
  4. From the Amazon Redshift cluster, use the cluster role and bucket role to run the UNLOAD command.

Important: This resolution doesn't apply to Amazon Redshift clusters or S3 buckets that use server-side encryption with AWS Key Management Service (AWS KMS).

Resolution

Create a bucket role

From the account of the S3 bucket, create an IAM role (bucket role) with permissions to the bucket:

  1. From the account of the S3 bucket, open the IAM console.

  2. Create an IAM role. As you create the role, select the following:
    For Select type of trusted entity, choose AWS service.
    For Choose the service that will use this role, choose Redshift.
    For Select your use case, choose Redshift - Customizable.

  3. After you create the IAM role, attach a policy that grants permission to the bucket:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Stmt1234537676482",
          "Action": [
            "s3:ListBucket",
            "s3:PutObject"
          ],
          "Effect": "Allow",
          "Resource": [
            "arn:aws:s3:::awsexamplebucket/*",
            "arn:aws:s3:::awsexamplebucket"
          ]
        }
      ]
    }
  4. Get the bucket role's ARN. You need the role's ARN for a later step.

Create a cluster role

From the account of the Amazon Redshift cluster, create another IAM role (cluster role) with permissions to assume the bucket role:

  1. From the account of the Amazon Redshift cluster, open the IAM console.

  2. Create an IAM role. When you create the role, select the following values:
    For Select type of trusted entity, choose AWS service.
    For Choose the service that will use this role, choose Redshift.
    For Select your use case, choose Redshift - Customizable.

  3. After you create the IAM role, attach the following policy to the role:

    Important: Replace arn:aws:iam::123456789012:role/Bucket_Role with the ARN of the bucket role that you created.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "Stmt1234537501110",
          "Action": [
            "sts:AssumeRole"
          ],
          "Effect": "Allow",
          "Resource": "arn:aws:iam::123456789012:role/Bucket_Role"
        }
      ]
    }
  4. Get the cluster role's ARN. You need the role's ARN for a later step.

Update the bucket role to create a trust relationship with the cluster role

  1. From the account of the S3 bucket, open the IAM console.

  2. In the navigation pane, choose Roles.

  3. From the list of roles, open the bucket role that you created.

  4. Choose the Trust relationships tab.

  5. Choose Edit trust relationship.

  6. For the Policy Document, replace the existing policy with the following:

    Important: Replace arn:aws:iam::012345678901:role/Cluster_Role with the ARN of the cluster role that you created.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "AWS": "arn:aws:iam::012345678901:role/Cluster_Role"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
  7. Choose Update Trust Policy.

From the Amazon Redshift cluster, run the unload operation

  1. Connect to the Amazon Redshift cluster.

  2. Run the UNLOAD command with both the IAM roles that you created, similar to the following example:

    Important: Replace arn:aws:iam::012345678901:role/Cluster_Role with the ARN of your cluster role. Then, replace arn:aws:iam::123456789012:role/Bucket_Role with the ARN of your bucket role.

    unload ('select * from TABLE_NAME')
    to 's3://awsexamplebucket' 
    iam_role 'arn:aws:iam::012345678901:role/Cluster_Role,arn:aws:iam::123456789012:role/Bucket_Role';

After you run the UNLOAD command, the same account owns the data files and the bucket that stores the files.

Related information

UNLOAD examples

AWS OFFICIAL
AWS OFFICIALUpdated 7 months ago
2 Comments

Hi, Can you please confirm this is still actual? I'm looking at the document here https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html

it appears by default Bucket owner enforced (default) – ACLs are disabled, and the bucket owner automatically owns and has full control over every object in the bucket. ACLs no longer affect permissions to data in the S3 bucket. The bucket uses policies to define access control.

replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied a year ago