Skip to content

Access Denied when copying S3 objects with boto3 even using admin permissions

0

Hi community,

I have a Lambda function (written with Python - boto3 ) that copies S3 objects to another folder in the same bucket - using s3_client.copy_object() . I want to reorganize the files so I can define partitioned tables in AWS Glue Data Catalog.

The problem is that even having admin permissions the lambda function returns this error: "An error occurred (AccessDenied) when calling the CopyObject operation: Access Denied"

Why does it happen if the role has admin permissions?

asked 2 years ago835 views

1 Answer
0

The "Access Denied" error you're encountering when trying to copy S3 objects using boto3 in your Lambda function, despite having admin permissions, could be due to a few reasons:

  1. S3 Object Lock: If the S3 bucket or the specific objects you're trying to copy have S3 Object Lock enabled, this could prevent the copy operation even with admin permissions. S3 Object Lock is designed to enforce retention policies and protect objects from being modified or deleted.

  2. Encryption: If the objects in your S3 bucket are encrypted using customer-provided keys (SSE-C) or AWS KMS keys, your Lambda function might not have the necessary permissions to decrypt the source object or encrypt the destination object. Even with admin permissions, you may need to explicitly grant access to the specific KMS keys used for encryption.

  3. Bucket Policy: There might be a bucket policy in place that's more restrictive and is overriding your IAM permissions. Bucket policies can deny actions even to users with full admin access if they're explicitly set to do so.

  4. Cross-Account Access: If you're trying to copy objects between buckets that belong to different AWS accounts, you might need additional permissions set up, even if your role has admin access in one account.

  5. VPC Configuration: If your Lambda function is configured to run inside a VPC, make sure it has the necessary network access to reach S3. This includes proper route table configurations and VPC endpoints for S3 if you're not using a NAT gateway.

  6. IAM Role Trust Relationship: Ensure that the trust relationship for your Lambda function's IAM role is correctly configured to allow Lambda to assume the role.

To troubleshoot this issue:

  1. Double-check your IAM role permissions and make sure they explicitly allow s3:GetObject for the source and s3:PutObject for the destination.

  2. Verify if S3 Object Lock is enabled on your bucket or objects.

  3. If using encrypted objects, ensure your Lambda function has permissions to use the relevant KMS keys.

  4. Review your bucket policies to make sure they're not restricting access.

  5. Use AWS CloudTrail to investigate the specific API calls and any permission-related issues.

  6. Consider using S3 presigned URLs as an alternative method to access the objects if direct access is being denied.

If the issue persists after checking these points, you may need to contact AWS support for further assistance, as there might be an account-specific configuration causing this behavior.
Sources
Tutorial: Transforming data for your application with S3 Object Lambda - Amazon Simple Storage Service
Security considerations for S3 Object Lambda Access Points - Amazon Simple Storage Service
Amazon Simple Storage Service (S3) — Cloud Storage — AWS

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.