Copied snapshot is being deleted after creation by DLM

Question

I'm using AWS Data Lifecycle Manager (DLM) to copy EBS snapshots across regions automatically. The snapshots are encrypted using the customer-managed keys with AWSDataLifecycleManagerDefaultRole. Everything appears to work at first — I can see a `CopySnapshot` event in CloudTrail in the target region — but then the copied snapshot never appears in the EC2 console or via CLI. And the source snapshot still exists in the source region.
After some investigation, I discovered that the snapshot is being created, but then automatically deleted around 13 minutes later. I confirmed this via CloudTrail — I see a `DeleteSnapshot` event shortly after the `CopySnapshot`.

I can confirm that the retention period is set correctly `{"Interval":1, "IntervalUnit": "MONTHS"}` for the copied snapshot.

**Some more context:**
* The snapshot copy is happening within the same AWS account, just across regions.
* The copied snapshot never becomes visible in the EC2 console or describe-snapshots output.
* There are no errors reported in the CopySnapshot or DeleteSnapshot CloudTrail events.
* I suspect DLM is deleting the copied snapshot, possibly due to retention settings, but the snapshot is being deleted.

Has anyone seen this behavior before? Is this a misconfiguration in my DLM policy?
Any guidance would be appreciated.

Riku_Kobayashi · Answer

Hello.

Who is executing the DeleteSnapshot event in the destination region?  
You can find out who is executing it by checking the CloudTrail event history.  
It may be possible that a service other than Data Lifecycle Manager is deleting the snapshots.  
You can check this in the "User name" column as shown below.  
![a](/media/postImages/original/IM6ZPY2WyURbOv_g6V8EcNbg)

re:Post Agent · Answer

This behavior appears to be related to how Amazon Data Lifecycle Manager (DLM) manages snapshots, particularly when there are issues with the snapshot creation or retention settings.

When DLM creates cross-region copies of snapshots, it should retain them according to your specified retention policy (in your case, 1 month). The fact that the snapshots are being deleted shortly after creation suggests there might be an issue with how DLM is managing these snapshots.

Here are some potential causes and solutions:

1. **Retention Schedule Type**: DLM supports two types of retention schedules - count-based and age-based. You've specified an age-based retention (`{"Interval":1, "IntervalUnit": "MONTHS"}`), which should keep the snapshots for one month. However, there might be an issue with how this is being applied to cross-region copies.

2. **Source Instance Termination**: If the source EC2 instance from which the original snapshot was taken has been terminated, this could affect how DLM manages the snapshots. According to DLM's behavior:
- For count-based retention schedules, if the source instance is terminated, DLM no longer manages the snapshots, but it doesn't automatically delete them.
- For age-based retention schedules (which you're using), DLM continues to delete snapshots on the defined schedule up to, but not including, the last snapshot.

3. **Encryption and Key Access**: Since you mentioned the snapshots are encrypted with customer-managed keys, verify that the AWSDataLifecycleManagerDefaultRole has proper permissions to use these keys in the target region.

4. **Monitoring and Debugging**: To better understand what's happening, you could:
- Set up an EventBridge rule to capture the DeleteSnapshot events with a pattern like:
```
{
"source": ["aws.ec2"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ec2.amazonaws.com"],
"eventName": ["DeleteSnapshot"]
}
}
```
- This could help you identify which service or role is initiating the deletion.

5. **Manual Testing**: Try creating a cross-region copy of the snapshot manually to see if it persists, which would help determine if the issue is specific to DLM.

If the issue persists, you might need to review your DLM policy configuration or contact AWS Support for further assistance, as this behavior doesn't align with the expected functionality of DLM's retention policies.  
**Sources**  
[Lifecycle Manger - Issue with Snapshot Management Post Instance Termination | AWS re:Post](https://repost.aws/questions/QUrl1hwDYIS1SMU_YQMjORbg/lifecycle-manger-issue-with-snapshot-management-post-instance-termination)  
[Amazon DLM for snapshot automation | AWS re:Post](https://repost.aws/questions/QU7IIT89-2TVeEXmsR4z9lxQ/amazon-dlm-for-snapshot-automation)  
[Is it possible to restore a deleted EC2 instance or snapshot from Lifecycle Backup | AWS re:Post](https://repost.aws/questions/QUItkaNP8BRUimnHhgcG-S5g/is-it-possible-to-restore-a-deleted-ec2-instance-or-snapshot-from-lifecycle-backup)

Copied snapshot is being deleted after creation by DLM

添加回答

相关内容