Skip to content

EBS Snapshots Clean-up Guidance

0

We have accumulated more than 300k EBS snapshots in our AWS account over the past 3 years due to an error in a daily-running script. For a single EBS volume, it has more than 1,000 snapshots.

We are trying to do the clean-up now with a custom script, but we are concerned about whether this will cause any performance impact to the associated volume or EC2 instance. Based on public documentation, I don't see any performance impact mentioned.

However, I would like to double-check this with the community given that the number of snapshots is huge. Also, I would like to seek any best practices in such a scenario to make it successful and faster.

asked 2 years ago939 views

2 Answers
0
Accepted Answer

Hi kj,

EBS snapshots are independently implemented and stored in S3. Deleting a snapshot will have no impact on the source EBS volumes or EC2 they are attached to. For the cleanup, it is recommended to delete the newest snapshot first, i.e. working backwards based on timestamps. If an older snapshot is deleted first, changed blocks from the previous snapshot must be consolidated into the next snapshot in the chain. If you delete the newest snapshot, there is nothing to consolidate. Therefore in theory it would be faster.

It is also recommended to use AWS Backup to manage EBS snapshots creation and retention with policy instead of custom script.

AWS

answered 2 years ago

EXPERT

reviewed 2 years ago

EXPERT

reviewed 2 years ago

0

Cleaning up a large number of EBS snapshots, such as in your case with over 300,000 snapshots accumulated over three years, generally does not cause direct performance impacts on the associated EBS volumes or EC2 instances. However, there are some considerations and best practices to keep in mind when undertaking such a large-scale cleanup:

  1. Snapshot Deletion Process: When you delete a snapshot, AWS only removes the data unique to that snapshot. This process happens in the background and doesn't affect the performance of your active volumes.

  2. Incremental Nature: EBS snapshots are incremental, meaning each snapshot only stores the blocks that have changed since the previous snapshot. Deleting a snapshot in the middle of a chain doesn't affect the ability to restore volumes from other snapshots in the chain.

  3. Automation: Given the large number of snapshots, it's advisable to use automation for the cleanup process. You can use AWS services like Amazon Data Lifecycle Manager or create custom scripts using the AWS CLI or SDK to manage the deletion process efficiently.

  4. Prioritization: Start by identifying and removing unnecessary snapshots, such as those from terminated instances or deleted volumes. You can use AWS Config rules or custom scripts to identify these.

  5. Retention Policy: Implement a snapshot retention policy to keep only the necessary snapshots. For instance, you might keep daily snapshots for a week, weekly snapshots for a month, and monthly snapshots for a year.

  6. Gradual Approach: Instead of deleting all unnecessary snapshots at once, consider a gradual approach. Delete snapshots in batches over time to minimize any potential impact on AWS infrastructure.

  7. Monitoring: Keep an eye on your AWS account's service limits and request limit increases if needed, as mass deletion might hit API rate limits.

  8. Cost Analysis: Before deletion, analyze the potential cost savings. Remember that deleting a snapshot doesn't necessarily reduce storage costs if other snapshots are referencing its data.

  9. Archiving: For snapshots that you want to keep for compliance or long-term storage but don't need quick access to, consider using the EBS snapshot archive feature. This can significantly reduce storage costs for infrequently accessed snapshots.

  10. Tagging: Implement a robust tagging strategy for future snapshots to make management and cleanup easier going forward.

By following these practices, you can efficiently clean up your excess snapshots while minimizing any potential risks or performance impacts. Remember to test your cleanup script on a small subset of snapshots before running it on the entire set to ensure it behaves as expected.
Sources
Cleaning up backups - AWS Prescriptive Guidance
Modify Amazon EBS snapshots - AWS Prescriptive Guidance
Guidelines and best practices for archiving Amazon EBS snapshots - Amazon EBS
Creating EBS volume backups with AMIs and EBS snapshots - AWS Prescriptive Guidance

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.