- Newest
- Most votes
- Most comments
Cross ref: https://repost.aws/questions/QUXB_IGAz7QnC-Iw2mKsNzew/options-for-deleting-s3-glacier-archives
Additionally, here is some sample code:
https://github.com/aws-samples/amazon-s3-glacier-archive-data-delete
Hope this helps!
If you want to empty a specific prefix ("folder name") or the entire bucket, the most general way to do it right while avoiding unexpected costs is to create a lifecycle rule in your bucket. You would filter it by prefix, if the whole bucket isn't to be emptied, and set the action to expire the objects. If the bucket is not versioning-enabled, that's all that is needed. If the bucket is versioning-enabled, you'll need one rule to make the "current" versions "past" versions first, and a second rule to cause the "past" versions to be deleted permanently.
Lifecycle rules start getting executed at midnight UTC time and may take many, many hours to complete. If you aren't dealing with astronomical numbers of objects, I would guess it would complete by the following evening (in UTC time).
If you want to be more selective than emptying an entire prefix/folder/bucket, such as only deleting objects in Deep Archive but not objects with the same prefix (in the same "folder") but in different storage classes, then the operation gets more complex. For example, you could create an S3 Batch Job to produce an inventory report, tag the objects based on their storage class, and then target the lifecycle rule based on the tag instead of a prefix. You'll want to note that inventory reports, batch jobs, and tagging requests are billable operations, although unless the numbers of objects are huge, they are rather inexpensive. It's best to check against the metrics for your bucket and the pricing calculator, though.
Lifecycle rules are explained in detail starting with this documentation article: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html, and "expiring" (=deleting objects/files or marking them as deleted) in more detail here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-expire-general-considerations.html
Thank you for the quick response. I'm new to AWS and haven't used lifecycle rules but can look through the documentation. I should be able to identify and download the few files we need to keep, and then eliminate the folder with the lifecycle rules.
I updated the text with links to the relevant parts of the documentation about lifecycle rules. You can find them by opening your S3 bucket in the console and clicking the "Management" tab. Lifecycle rules are configured at the top of the screen.
Relevant content
- asked a year ago
- asked 2 years ago
- asked 5 months ago
- asked a year ago
AWS OFFICIALUpdated 3 years ago
Thank you for the quick response.
To be clear, those links are not about S3 objects in the Glacier Deep Archive storage class. They are about the separate S3 Glacier service, which uses a completely different set of APIs from the regular S3 objects in the Deep Archive storage class discussed here. Most users should regard the S3 Glacier service as a legacy remnant they should never touch.