Best way how to exclude a specific prefix in S3 replication rule

Question

Hi, I am searching for best way how to **exclude** a specific prefix in S3 replication rule and am not able to find a suitable answer in AWS documentation.

**Requirements:**   
* Environment is a multi-account setup with help of AWS Control Tower, aggregating all logs from member accounts into a central Logging account
* Control Tower stores CloudTrail and Config in one bucket, but in (from my point of view) not good structure
* With help of S3 replication rule, CloudTrail and Config shall be split into two separate buckets for better log processing

**Issue:**   
* I am unable to find a way how to **exclude** a specific prefix in S3 replication rule
* From S3 replication settings, there is only an "allow" prefix configuration option. As we will have at least dozens of AWS accounts in our environment, maintaining this prefix-"allow-list" is not suitable
* From other re:post I understood tagging would be an option, which would require a solution on how to tag all objects
* My idea was to restrict the IAM role that is being used for S3 replication rule with a "DENY" for specific folder/prefix, e.g.
* ALLOW {bucketname}/{org-id}/AWSLogs/{account#}/Config/*
* DENY {bucketname}/{org-id}/AWSLogs/{org-id}/{account#}/CloudTrail/*
* The deny would be of course after the "AWSLogs/{org-id}/", but above is more illustrative. For CloudTrail there would be a second S3 replication rule. Unfortunately an "IAM role applies to all rules in the bucket. To change the IAM role for all rules in the bucket, edit the Replication configuration settings."

Looking for suitable way on how to **exclude** a specific prefix in S3 replication rule. I assume restricting the IAM role will lead to lots of deny logs, because the S3 replication rule will still be triggered, but IAM role will not be allowed to replicate CloudTrail. Happy for any input.

Answer

Hello Andre. 
In my opinion you can do two other things: 
**Tagging Objects:**
Tag all the objects in the source bucket with metadata indicating whether they should be replicated or not.
For example, you can tag CloudTrail logs with a tag like "Replicate: Yes" and Config logs with "Replicate: No."

**IAM Policy for Replication:**
Create an IAM policy for the IAM role used by S3 replication that allows replicating objects based on their tags. This policy should use the s3:ReplicateTags condition key.
Example IAM Policy for Replication Role:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ReplicateObject",
            "Resource": "arn:aws:s3:::destination-bucket/*",
            "Condition": {
                "StringEquals": {
                    "s3:ExistingObjectTag/Replicate": "Yes"
                }
            }
        }
    ]
}
```

Best regards, Andrii

Answer

Just an update on this topic: I did lots of investigation and the request is simply not possible in AWS at this time.
What did we do? We disabled default CloudTrail from AWS Control Tower and configured our own Organizational Trail.

Answer

I took Andrii's comment as input to search for tagging automation solution and we found some good starting point (if you are interested, search for "greg heywood tagging automation"). We adjusted his Lambda script to tag the config objects only and exclude the cloud trail logs. With tagged config logs we can now use s3 replication based on tags.

An out of the box solution to exclude prefixes would have been preferred, but this one is working too.

Best way how to exclude a specific prefix in S3 replication rule

Relevant content