Lifecycle Configuration Standard --> Standard IA -- Glacier Flexible Restore via CloudFormation
We do shared web hosting and my cPanel servers stores backups in S3, each server with its own bucket. cPanel does not have a provision to select the storage class, so everything gets created as Standard. With around 9TB of backups being maintained, I would really like them to be stored as Standard IA after the first couple of days, and then transition to Glacier after they have been in IA for 30 days. The logic here is the backup that is most likely needed would be the most recent. Currently we skip the step of transferring to IA and they go straight to Glacier after 30 days.
According to this page, that kind of multi staged transition should be ok, and it confirms that the transitions from class to class I want are acceptable. https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html
The examples on this page show a transition in days of 1, seeming to show that a newly created object stored in Standard can be transitioned immediately: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-lifecycleconfig.html
My YAML template for Cloud Formation has this section in it:
- Id: TransitionStorageType
Status: Enabled
Transitions:
- StorageClass: "STANDARD_IA"
TransitionInDays: 2
- StorageClass: "GLACIER"
TransitionInDays: 32
When I run the template all of the buckets update with nice green check marks, then the whole stack rolls back without saying what the issue is. If turn that into 2 separate rules like this:
- Id: TransitionStorageIA
Status: Enabled
Transitions:
- StorageClass: "STANDARD_IA"
TransitionInDays: 2
- Id: TransitionStorageGlacier
Status: Enabled
Transitions:
- StorageClass: "GLACIER"
TransitionInDays: 32
Then each bucket getting modified errors with: Days' in Transition action must be greater than or equal to 30 for storageClass 'STANDARD_IA'
but if you look at the rules, it is in Standard IA for 30 days as it doesn't change to Glacier until day 32, and it transitions to Standard IA at day 2. So that error does not make any sense.
What do I need to do to make this work? My monthly bill is in serious need of some trimming. Thank you.
So the devil is in the details, as always. You can transition from Standard to Glacier from day zero, but if you are transitioning to Standard IA or One Zone IA, then there is a 30 day limit imposed. I thought this had me stuck without a solution, until I noticed how minuscule the pricing difference is between Glacier Instant Retrieval and Glacier Flexible Retrieval.
This is the Lifecycle policy I ended up going with:
LifecycleConfiguration:
Rules:
- Id: AbortIncompleteMultipartUpload
Status: Enabled
AbortIncompleteMultipartUpload:
DaysAfterInitiation: 7
- Id: RetentionPolicy
Status: Enabled
Transitions:
- StorageClass: "GLACIER_IR"
TransitionInDays: 7
ExpirationInDays: 97
I hope this can help someone in the future if they have a similar issue.
Relevant questions
Virtual hosting an S3 bucket using Cloudfront + SSL
asked 3 years agoCan you determine what AZ an S3 One Zone IA bucket is stored in?
Accepted Answerasked 3 years agos3 lifecycle transition costs
Accepted Answerasked 2 years agoS3 Glacier (2013) Flexible running without Bucket, want to move to Glacier Deep Archive
asked 2 months agoWill we allow transitioning between Glacier and Glacier Deep Archive?
Accepted Answerasked 4 years agoLifecycle Configuration Standard --> Standard IA -- Glacier Flexible Restore via CloudFormation
Accepted Answerasked a month agoTape Gateway on Amazon S3 Standard Storage
Accepted Answerasked 2 years agoNew to AWS, my lifecycle rule seems to not be applied (Versioned bucket)
asked a year agoExpire S3 glacier deep archive using lifecycle management not working
Accepted Answerasked 5 months agoDoes changing a s3 lifecycle policy change all older objects in an s3 bucket?
asked 2 months ago