Why is my EC2 AMI or EBS snapshot creation slow?

Short description

Amazon EBS-backed AMIs include one or more Amazon EBS snapshots. The creation of EBS-backed AMIs or EBS snapshots might be slow because of the large amount of data that must be copied to Amazon Simple Storage Service (Amazon S3). Many factors, such as Write activity on the EBS volume, can impact the creation time. Therefore, creation times for the snapshots might vary widely.

Resolution

Dirty blocks

The most common cause for slow creation of AMIs or snapshots is the amount of dirty data that must be copied to Amazon S3. This dirty data is measured by the number of blocks. The following factors might result in a large number of dirty blocks:

• Size of the EBS volume
• Time since last snapshot
• Write activity on the volume

Snapshots are designed to be incremental. This means that Amazon EBS copies only the blocks that changed since the last snapshot was created. An EBS volume might not have an existing snapshot because no snapshots were created before, or previous snapshots were deleted. In such cases, the blocks can't be compared against any snapshot. Therefore, all the blocks are considered dirty.

Also, if a snapshot was created a long time ago, or the EBS volume is very active, you might have a large number of blocks that must be copied as part of the new snapshot.

To avoid this issue, it's a best practice to create snapshots frequently. With this practice, the number of blocks to be copied for each snapshot is smaller. You can use Amazon Data Lifecycle Manager to automate the creation, retention, and deletion of snapshots for your EBS volumes. You can set the frequency of snapshots based on your Recovery Point Objective (RPO). EBS snapshots aren't charged based on the number of snapshots, but rather for the incremental data saved in S3. For more information, see Amazon EBS pricing.

Multiple volumes queued for snapshot creation

Snapshot creation is a shared bandwidth operation. This means that Amazon EBS uses a shared bandwidth to send data to S3. Therefore, you might encounter delays if there are multiple volumes queued for snapshot creation. This typically happens if several snapshots are created at the top of the hour. For example, if you have automated processes to create snapshots exactly at midnight, then the snapshot creation might be delayed.

To avoid this issue, it's a best practice to create snapshots at various times past the hour. This practice might help reduce the time taken to create a snapshot. You can use Amazon Data Lifecycle Manager to automatically create snapshots within an hour after the scheduled start time, instead of creating snapshots immediately.

Stacked snapshots

When you create multiple snapshots for the same volume within a short period of time, then the first snapshot is created, and the other snapshots are moved to the Pending state. The creation of the snapshots in the Pending state doesn't progress until the first snapshot creation is completed. Deleting a snapshot that's in the Pending state doesn't stop the creation process. If you try to delete the snapshot that's in the Pending state, the snapshot is created first before it's deleted.

To resolve this issue, avoid creating snapshots when a snapshot creation for a volume is in progress, unless it's required to do so.

Related information

My Amazon EBS snapshots show no progress or receive the error "ConcurrentSnapshotLimitExceeded". How can I resolve this?