Export Elasticache backup to s3 using boto3

Question

Export working from console, but when trying to export Elasticcache backup to s3 using boto3 (from lambda function) I am getting the following error:

```
[ERROR] InvalidParameterValueException: An error occurred (InvalidParameterValue) when calling the CopySnapshot operation: Elasticache was unable to validate the authenticated user has access on the S3 bucket BACKUP_NAME.
```
The bucket ACL configured correctly according to this [guide](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-exporting.html)

Python boto3:
```
ec_client = boto3.client('elasticache')
response = ec_client.copy_snapshot(
                SourceSnapshotName=source_snapshot_name,
                TargetSnapshotName=target_snapshot_name,
                TargetBucket=s3_bucket_name
            )
```
Lambda role's permissions:

```
{
    "Statement": [
        {
            "Action": [
                "elasticache:DescribeSnapshots",
                "elasticache:CopySnapshot"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:GetBucketAcl",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:ListAllMyBuckets"
            ],
            "Effect": "Allow",
            "Resource": [
                "BUCKET_NAME/*",
                "BUCKET_NAME"
            ]
        }
    ],
    "Version": "2012-10-17"
}
```

Bucket ACL:

```
External account
Canonical ID:
540804c33a284a299d2547575ce1010f2312ef3da9b3a053c8bc45bf233e4353     Objects: List, Write  Bucket ACL: Read, Write
```

Ron · Accepted Answer

Apparently the `copy_snapshot` method in boto3 requires the permission to list all buckets. This issue was resolved by updating the Lambda role's permissions with the following statement:

```
{
    "Action": [
        "s3:ListAllMyBuckets"
    ],
    "Effect": "Allow",
    "Resource": [
        "arn:aws:s3:::*"
    ]
}
```

This statement grants the Lambda role permission to list all buckets in Amazon S3.

vtjean · Answer

You're missing the s3:DeleteObject permission on your Lambda IAM Role, check the following documentation to validate your permissions:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-exporting.html#backups-exporting-procedures

I also recommend verifying the s3 bucket policy:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-exporting.html#backups-exporting-grant-access:~:text=Pacific%20(Hong%20Kong)-,Region.),-%7B%0A%20%20%20%20%22Version%22%3A%20%222012

Ron · Answer

Same error even with s3:*.
I believe the problem is related to ACL permissions because the same error occurred when I tried to export the snapshot via the console before I configured the bucket ACL as described in the [documentation](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-exporting.html#backups-exporting-grant-access).
But now everything is configured as it should. Maybe some kind of integration problem with lambda or boto3?

Cloudtrail access denied event:

```
{
    "eventVersion": "1.09",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "PRINCIPAL_ID:ROLE_NAME",
        "arn": "arn:aws:sts::ACCOUNT_ID:assumed-role/ROLE_NAME/ROLE_NAME",
        "accountId": "ACCOUNT_ID",
        "accessKeyId": "ACCESS_KEY_ID",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "PRINCIPAL_ID",
                "arn": "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME",
                "accountId": "ACCOUNT_ID",
                "userName": "ROLE_NAME"
            },
            "attributes": {
                "creationDate": "2024-01-31T08:28:02Z",
                "mfaAuthenticated": "false"
            }
        },
        "invokedBy": "ec.amazonaws.com"
    },
    "eventTime": "2024-01-31T08:28:04Z",
    "eventSource": "s3.amazonaws.com",
    "eventName": "ListBuckets",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "ec.amazonaws.com",
    "userAgent": "ec.amazonaws.com",
    "errorCode": "AccessDenied",
    "errorMessage": "Access Denied",
    "requestParameters": {
        "Host": "s3.us-east-1.amazonaws.com"
    },
    "responseElements": null,
    "additionalEventData": {
        "SignatureVersion": "SigV4",
        "CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
        "bytesTransferredIn": 0,
        "AuthenticationMethod": "AuthHeader",
        "x-amz-id-2": "x-amz-id-2",
        "bytesTransferredOut": 243
    },
    "requestID": "4AFDSGDSFGFSQKWN",
    "eventID": "3b64567-1da4-4c42-846c-80c35546ea14",
    "readOnly": true,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "ACCOUNT_ID",
    "vpcEndpointId": "VPC_ENDPOINT_ID",
    "eventCategory": "Management"
}
```

Export Elasticache backup to s3 using boto3

Relevant content