S3 bucket life cycle configuration overlapping rules

Question

I have a usecase wherein I need to apply a lifecycle configuration rule to my s3 bucket. Now, what I want to do after 90 days all the objects inside the bucket should be moved to glacier storage. I know how to write the rule for that and it is straightforward. yes, I'm using terraform.

```
  rule {
    id = "send-to-glacier"  // glacerize entire s3 bucket
    transition {
      days = 90
      storage_class = "GLACIER"
    }
    status = "Enabled"
  }
```

Now I want to exclude objects inside a particular `subFolder` inside the same bucket from being converted to glacier storage class. 
* is the below rule correct? Will this achieve my goal?
* Since this subFolder has conflicting rules, does the order in which the rules are written decides the precendence of the rules?
* If below rule is not correct, can someone please suggest solution?

```
  rule {
    id = "keep-in-standard"  // do not glacerize this postalData subfolder
    filter {
      prefix = "subFolder/"
    }

expiration {
      days = 3650000 // really high value
    }

status =  "Enabled"
  }
```
I can't test things out right now as the transition happens at midnight utc. Also this particular test case is not mentioned in [docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-conflicts.html#lifecycle-config-conceptual-ex5).

re:Post Agent · Answer

Hello,

Thanks for reaching out to us at AWS re:Post. Sameer this side from S3 Support Team and here to assist further on the question you asked here.

As from the post, I understand that you would like to setup a S3 bucket lifecycle rule to move all the objects to Glacier storage class 90 days and want to exclude a specific subfolder from this action (do not wish to move data in this subfolder to Glacier storage class). Please correct me if I have misunderstood your query.

I would like to share that the use-case to exclude a subfolder/s is not currently supported with S3 Lifecycle rules. You can currently use the below filters [1] :
    - Apply rule to the whole S3 bucket 
    - Apply rule to a specific prefix (one prefix can be specified per lifecycle rule) 
    - Apply rule based on the Object size
    - Apply rule based on the Object tags.

As there is no direct solution to your use-case, you can use any of the below workarounds to setup lifecycle rule on the S3 bucket. For this, let us consider you have 10 folders at the root of your S3 bucket and would like to exclude 1 folder in them :

1. Apply prefix-based lifecycle rule on all the 9 subfolders except the one specific subfolder which you would like to exclude.

2. If the objects present in 9 subfolders have any specific pattern — contain any specific tags — use a tag-based lifecycle rule.

3. If the objects present in 1 subfolder which you would like to exclude have a pattern of their size — either are all large files/small files as compared to the objects in other 9 subfolders — use a size-based filter in the lifecycle rule.

5. Move all the 9 subfolders inside another folders at the root of the bucket and apply a prefix-based lifecycle rule on that new folder created. OR, move the 1 subfolder to a different S3 bucket or to a different root-level folder in the same bucket.

I hope I was able to address your query on the post. In case of any follow-up queries/clarification, please proceed to open a case with AWS Support Engineering at [3]. Thanks for your patience & cooperation!

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html 
[2] https://docs.aws.amazon.com/AmazonS3/latest/userguide/intro-lifecycle-filters.html 
[3] https://console.aws.amazon.com/support/home#/case/create?issueType=technical

S3 bucket life cycle configuration overlapping rules

Add your answer

Relevant content