From Lambda, duplicate a S3 folder (object) tree

0

I am trying to duplicate a tree (2 levels) of objects in executing the Lambda. I have searched online and found many examples using boto3 & s3pathlib. I've made sure that the IAM account executing the Lambda created the S3 bucket and folders. I've made sure that the IAM account has the correct permissions. I've even added IAM permissions for executing Lambda as described in https://repost.aws/knowledge-center/lambda-copy-s3-files.

Regardless of any of the examples, the execution fails. Not in the lambda code but deeper in a library....the error is the same regardless of what Lambda function I try - I've written 5 of them.

Any help would be appreciated! I'm stuck.

  • Can you update your answer to include the actual error you are getting...

Petrus
asked 2 months ago66 views
1 Answer
0

Tricky... please try the following: Make sure the IAM role used by the Lambda function has the necessary permissions to access the source and destination buckets. This includes s3:GetObject and s3:PutObject permissions.

When copying objects, the ownership does not automatically transfer to the destination bucket account. You need to explicitly grant the destination bucket owner permissions to the copied objects.

Try copying an object while setting the ACL to grant the destination bucket full control. For example:

aws s3 cp s3://source-bucket/object s3://destination-bucket/object --acl bucket-owner-full-control Verify the bucket policy on the destination bucket allows the Lambda execution role to put objects.

Check for any explicit Deny policies that may be blocking access.

profile picture
EXPERT
answered 2 months ago
  • This creates more questions...but first some answers for you. Thanks in advance....

    Just checked - The IAM DOES HAVE permissions for s3:GetObject & s3:PutObject.

    Your example: aws s3 cp s3://source-bucket/object s3://destination-bucket/object --acl bucket-owner-full-control Are you suggesting this is placed in the Lambda Python function I wrote? Is this statement in Python?

    You also say: Verify the bucket policy on the destination bucket allows the Lambda execution role to put objects.

    You also say: Check for any explicit Deny policies that may be blocking access. Where and how to do this?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions