How can I troubleshoot problems using Amazon Data Lifecycle Manager?

5 minute read
0

My Amazon Data Lifecycle Manager policy is in an error state, or does not act as expected regarding snapshots. How can I troubleshoot these issues?

Short description

The following are common reasons that your lifecycle policy is in an error state, or fails to create or copy Amazon Elastic Block Store (Amazon EBS) snapshots:

  • The lifecycle policy isn't turned on.
  • There are incorrect permissions on the policy.
  • You're using an AWS Identity and Access Management (IAM) role other than the default AWSDataLifecycleManagerDefaultRole, and there are issues with trust relationships.
  • There are duplicate tags on the policy.
  • A tag defined in the policy is already in use.
  • Your resources are encrypted.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, then make sure that you’re using the most recent AWS CLI version.

Snapshots aren't created as expected

If snapshots aren't created, then verify that the lifecycle policy is turned on.

1.    Open the Amazon Elastic Compute Cloud (Amazon EC2) console.

2.    Select Lifecycle Manager, and then verify that the policy State is set to ENABLED.

3.    If the policy isn't set to ENABLED, then choose Actions, Modify Snapshot Lifecycle Policy.

Note: If the policy state is ERROR, see the following section, DLM policy is in the ERROR state.

4.    Select Enable policy, and then select Update policy.

Note: It can take up to an hour after creating a lifecycle policy for snapshots to be created. After at least an hour has passed, open the Amazon EC2 console. Then, select Snapshots to verify that snapshots are being created.

Unable to copy snapshots between Regions

If the copied snapshot copied is encrypted, then the user must have access to the source and destination AWS Key Management Service (AWS KMS) key. For more information, see Determining access to an AWS KMS key.

The lifecycle policy is in an error state

A lifecycle policy in the error state can be caused by one or more of these issues:

  • There is a problem with your resource tags.
  • The Amazon Data Lifecycle Manager permissions aren't correct.
  • The IAM permissions aren't correct.

In addition, if you're using a custom IAM role, a trust relationship might not be attached to the role.

View information about what caused the error state by checking Amazon CloudWatch Events. The following are common errors and resolutions:

Duplicate tag key

If there are duplicate tags in your lifecycle policy, then a CloudTrail Event similar to the following appears. In the following example, the tag key Name is duplicated in the policy.

CreateSnapshot @2018-12-24T20:25:58.000Z UTC
"errorCode": "Client.InvalidParameterValue",
 "errorMessage": "Duplicate tag key 'Name' specified.", 
    "requestParameters": {
        "volumeId": "vol-xxxxxxxxxxxx",
        "description": "Created for policy: policy-xxxxschedule: First Schedule",

1.    Open the Amazon EC2 console.

2.    Select Lifecycle Manager.

3.    Select your lifecycle policy, and then choose Actions, Modify Lifecycle Policy.

4.    In the Tag created EBS snapshots section, change the Key on the duplicated tag to a unique name.

5.    Select Update policy.

Tag (Name) is already defined in resource id vol-xxxxxxxxxxxx

If a tag that's defined in your lifecycle policy is already in use in a different lifecycle policy, then you might have an issue if:

  1. The lifecycle policy is in the same account, and
  2. The lifecycle policy is for the same resource.

In this case, a CloudTrail Event similar to the following appears:

CreateSnapshots
---------------------------------------------------------------------------------
    "eventVersion": "1.05",
    "userIdentity": {
        "type": "AssumedRole",
   "eventTime": "2020-01-xxxxxxxx",
    "eventSource": "ec2.amazonaws.com",
    "eventName": "CreateSnapshots",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "dlm.amazonaws.com",
    "userAgent": "dlm.amazonaws.com",
    "errorCode": "Client.InvalidParameterCombination",
    "errorMessage": "Tag (Name) is already defined in resource id vol-xxxxxxxx.",
    "requestParameters": {
"requestParameters": {
        "CreateSnapshotsRequest": {
            "Description": "Created for policy: policy-xxxxxxxschedule: Default Schedule",
            "InstanceSpecification": {
                "ExcludeBootVolume": false,
                "InstanceId": "i-xxxxxxx"
            },

A volume or instance can have more than one policy associated with it, but tags can't be duplicated across policies. For more information, see Considerations for Amazon Data Lifecycle Manager.

To correct this error, do the following:

1.    View your lifecycle policies to determine which tag is duplicated.

2.    Create a new lifecycle policy using a different tag, or edit your current lifecycle policy to use a different tag.

Client.AuthFailure

The "Client.AuthFailure" error might occur if the custom lifecycle policy or the IAM user don't have permissions set correctly. The following is an example of a Client.AuthFailure caused by an inaccessible key:

"Client.AuthFailure",
"errorMessage": "The specified keyId
arn:aws:kms:us-west-1:xxxxxxxxxxxxx:key/4ad6a1d7-53ac-45a3-8f08-e6eccc948fdd is not accessible",

For instructions on setting permissions for Amazon Data Lifecycle Manager, see Permissions for Amazon Data Lifecycle Manager.

For instructions on setting permissions for IAM users to use Amazon Data Lifecycle Manager, see Permissions for IAM users.


Related information

Amazon Data Lifecycle Manager API reference

AWS CLI Command reference - dlm

AWS OFFICIAL
AWS OFFICIALUpdated a year ago