How do I fix the "Unable to validate the following destination configurations" error in CloudFormation?

11 minute read

I subscribed to an AWS service and received the "Unable to validate the following destination configurations" error in AWS CloudFormation.

Resolution

To resolve the Unable to validate error, take the troubleshooting actions for your configuration.

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Issues with Lambda notification configuration

Lambda function ARN doesn't exist or isn't valid

If your Amazon Simple Storage Service (Amazon S3) bucket uses the AWS Lambda property LambdaConfigurations, then you might receive the lambda function does not exist error. This error occurs if the ARN configured for the Lambda function in the CloudFormation template doesn't exist or isn't valid.

To check whether the Lambda function exists, run the get-function command:

aws lambda get-function 
--function-name YOUR-FUNCTION-ARN-VALUE

Note: Replace YOUR-FUNCTION-ARN-VALUE with the function's ARN.

If you receive an error in the command output, then the function's ARN isn't valid or doesn't exist. Update the template to include the correct ARN. Then, create a new stack with the updated template, or update the existing stack.

Lambda function doesn't have permission to invoke Amazon S3

If you receive the resource you requested does not exist error, then your Lambda function is missing a required permission. To resolve this issue, complete the following steps:

To check the Lambda function permissions, run the get-policy command:

aws lambda get-policy 
--function-name YOUR-FUNCTION-ARN-VALUE 
--region YOUR-REGION

Note: Replace FUNCTION-ARN-VALUE with the function's ARN and YOUR-REGION with your AWS Region.

To allow the Lambda function to invoke Amazon S3, update the CloudFormation template to attach the following permissions:

S3Permission:    
  Type: AWS::Lambda::Permission
  Properties:
    FunctionName: YOUR-FUNCTION-ARN-VALUE
    Action: lambda:InvokeFunction
    Principal: s3.amazonaws.com
    SourceAccount: !Ref 'AWS::AccountId'

Note: Replace FUNCTION-ARN-VALUE with the function's ARN and AccountID with the AWS account that owns the function. You can use the AWS::AccountId pseudo parameter to automatically replace the account ID where CloudFormation creates the stack.

To make sure that CloudFormation creates the S3 bucket only after the Lambda function has the required permissions, add the DependsOn attribute:

S3Bucket:    
  Type: AWS::S3::Bucket
  DependsOn: "S3Permission"
  Properties:
    NotificationConfiguration:
      LambdaConfigurations:
        - Function: YOUR-FUNCTION-ARN-VALUE
            Event: "s3:ObjectCreated:Put"

Note: Replace FUNCTION-ARN-VALUE with your function's ARN.

Issues with Amazon SNS notification configuration

Amazon SNS ARN doesn't exist or isn't valid

If your S3 bucket uses the TopicConfigurations property, then you might receive the Unable to validate the following destination configurations error. This error occurs when the Amazon Simple Notification Service (Amazon SNS) topic doesn't exist or isn't valid. The ARN format and value must match the SNS topic's ARN.

To check whether the SNS topic's ARN exists in your account, run the list-topics command:

aws sns list-topics \
--region YOUR-REGION \
--query "Topics[?TopicArn=='YOUR-TOPIC-ARN-VALUE']"

Note: Replace YOUR-REGION with your Region and YOUR-TOPIC-ARN-VALUE with the topic's ARN.

If you don't receive any records in the command output, then the SNS topic either doesn't exist or isn't valid. To resolve this issue, create the SNS topic. Make sure that you provide a valid topic ARN in the TopicConfigurations property.

SNS topic doesn't have a required access policy

To verify that the SNS topic has the required access policy, complete the following steps:

To check the access policy that's attached to the SNS topic, run the get-topic-attributes command:

aws sns get-topic-attributes \
--topic-arn YOUR-TOPIC-ARN-VALUE \
--region YOUR-REGION \
--query 'Attributes.Policy'

Note: Replace YOUR-TOPIC-ARN-VALUE with your topic ARN and YOUR-REGION with your Region.

The access policy must allow the Amazon S3 service to publish to the topic. If the policy doesn't have those permissions, then edit the topic's access policy to include the following permissions:

{     
      "Sid": "S3AccessForNotification",      
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "YOUR-TOPIC-ARN-VALUE"
}

Note: Replace YOUR-TOPIC-ARN-VALUE with your topic ARN.

Verify that you can now create a new stack, or update the existing stack.

Issue with the AWS KMS key policy that's associated with the SNS topic

The AWS Key Management Service (AWS KMS) policy must allow Amazon S3 to access the AWS KMS key. To view the encryption configuration and the minimum required policy, complete the following steps:

To check whether the SNS topic is encrypted with an AWS KMS key, run the get-topic-attributes command:

aws sns get-topic-attributes \
--topic-arn YOUR-TOPIC-ARN-VALUE \
--region YOUR-REGION \
--query "Attributes.KmsMasterKeyId"

Note: Replace YOUR-TOPIC-ARN-VALUE with your topic ARN and YOUR-REGION with your Region. If the SNS topic is encrypted, then the command's output shows the AWS KMS key ARN.

To check the AWS KMS key policy, run the get-key-policy command:

aws kms get-key-policy \
--key-id YOUR-KMS-KEY-ARN \
--policy-name default \
--region YOUR-REGION | jq -r '.Policy' | jq .

Note: Replace YOUR-KMS-KEY-ARN with your AWS KMS key ARN, and YOUR-REGION with your Region. For better readability, it's a best practice to use the jq command to show the policy contents in a JSON format. For more information, see ./jq on the jq website.

The following example policy shows the minimum required AWS KMS key policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": ["kms:GenerateDataKey*", "kms:Decrypt"],
            "Resource": "YOUR-KMS-KEY-ARN"
        }
    ]
}

Note: Replace YOUR-KMS-KEY-ARN with your AWS KMS key ARN.

If your policy doesn't have the required permissions, then update the AWS KMS key policy.

Issues with Amazon SQS notification configuration

The Amazon SQS ARN doesn't exist or isn't valid

If your S3 bucket uses the QueueConfigurations property, then you might receive the SQS queue does not exist error. This error occurs when the Amazon Simple Queue Service (Amazon SQS) ARN doesn't exist or isn't valid.

To check whether the Amazon SQS queue exists in the account, run the list-queues command:

aws sqs list-queues 
--queue-name-prefix YOUR-SQS-QUEUE-NAME 
--region YOUR-REGION

Note: Replace YOUR-SQS-QUEUE-NAME with your SQS queue name, and YOUR-REGION with your Region.

If the SQS queue doesn't exist, then either create a new queue or update the template with an existing queue:

Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
          - Sid: Statement-id
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: "SNS:Publish"
            Resource: !Ref SNSTopic
            Condition:
              ArnLike:
                aws:SourceArn: !Join
                  - ''
                  - - 'arn:aws:s3:::'
                    - !Ref S3Bucket
      Topics:
        - !Ref SNSTopic
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: BucketOwnerFullControl

SQS queue doesn't have the required access policy

To verify that the SQS queue has the required access policy, you can use the AWS CLI or the CloudFormation console.

To use the AWS CLI to verify your queue's access policy, complete the following steps:

To check the SQS queue policy, run the get-queue-attributes command:

aws sqs get-queue-attributes \
--queue-url YOUR-SQS-QUEUE-URL \
--region YOUR-REGION \
--attribute-names Policy | jq .

Note: Replace YOUR-SQS-QUEUE-NAME with your SQS queue name, YOUR-REGION with your Region, and YOUR-SQS-QUEUE-URL with your queue URL. For better readability, it's a best practice to use the jq command to show the policy contents in a JSON format. For more information, see ./jq on the jq website.

If your policy doesn't have access to Amazon S3, then create a new access policy in a JSON file:

{"Policy": "{\"Version\":\"2012-10-17\",\"Statement\":
[{\"Sid\":\"S3AccessForNotification\",\"Effect\":\"Allow\",\"Principal\":
{\"Service\":\"s3.amazonaws.com\"},\"Action\":\"SQS:SendMessage\",\"Resource\":\"YOUR-SQS-QUEUE-ARN\"}]}"}

Note: Replace YOUR-SQS-QUEUE-ARN with the queue ARN.

To update the policy, run the set-queue-attributes command:

aws sqs set-queue-attributes 
--queue-url YOUR-SQS-QUEUE-URL 
--attributes file://sqs-policy.json

Note: Replace YOUR-SQS-QUEUE-URL with your queue URL and sqs-policy.json with your policy JSON file.

To use the CloudFormation console to verify your queue's access policy, update the CloudFormation template to include the following resource:

SampleSQSPolicy: 
  Type: AWS::SQS::QueuePolicy
  Properties: 
    Queues: 
      - YOUR-SQS-QUEUE-URL
    PolicyDocument: 
      Statement: 
        - 
          Action: 
            - "SQS:SendMessage"
          Effect: "Allow"
          Resource: YOUR-SQS-QUEUE-ARN
          Principal:  
            Service: 
              - "s3.amazonaws.com"

Note: Replace YOUR-SQS-QUEUE-URL with your queue URL, and YOUR-SQS-QUEUE-ARN with your queue ARN.

After you create or update the policy, verify that you can now create a new stack, or update the existing stack.

For more information, see What permissions do I need to access an Amazon SQS queue? If you still encounter issues, then see How do I troubleshoot the Amazon SQS error "Invalid value for the parameter policy"?

Issue with the AWS KMS key policy associated with the SQS queue

To resolve issues with the AWS KMS key policy, complete the following steps:

To get the SQS queue URL, run the list-queues command:

aws sqs list-queues 
--queue-name-prefix YOUR-SQS-QUEUE-NAME 
--region YOUR-REGION

Note: Replace YOUR-SQS-QUEUE-NAME with your SQS queue name and YOUR-REGION with your Region.

To get the AWS KMS key ID, run the get-queue-attributes command:

aws sqs get-queue-attributes \
--queue-url YOUR-SQS-QUEUE-URL \
--region YOUR-REGION \
--attribute-names KmsMasterKeyId

Note: Replace YOUR-SQS-QUEUE-URL with your queue URL, and YOUR-REGION with your Region.

To get the AWS KMS key ARN, run the describe-key command:

aws kms describe-key --key-id YOUR-KMS-KEY-ID

Note: Replace YOUR-KMS-KEY-ID with the key ID.

To view the full key policy, run the get-key-policy command:

aws kms get-key-policy \
--key-id YOUR-KMS-KEY-ARN \
--policy-name default \
--region YOUR-REGION

Note: Replace YOUR-KMS-KEY-ARN with your AWS KMS key ARN, and YOUR-REGION with your Region.

In the output, verify that the policy allows Amazon S3 to use the AWS KMS key. Example permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "s3.amazonaws.com"
            },
            "Action": ["kms:GenerateDataKey*", "kms:Decrypt"],
            "Resource": "YOUR-KMS-KEY-ARN"
        }
    ]
}

Note: Replace YOUR-KMS-KEY-ARN with the AWS KMS key's ARN.

If your policy doesn't have the required permissions, then update the key policy.

Circular dependency between resources

Important: Before you subscribe an SNS topic to S3 Event Notifications, you must create the AWS::SNS::TopicPolicy with the required permissions. The topic policy must exist before you create the subscription.

To create the topic policy first, you must use a DependsOn attribute on the AWS::S3::Bucket resource. This attribute creates the topic policy before the bucket. Or, you can use two stack operations to create all resources first, and then update the S3Bucket resource to include the NotificationConfiguration property. Take one of the following actions.

Specify a value for BucketName in your CloudFormation template

Use a static name for your S3 bucket in the BucketName property in the S3Bucket resource of your CloudFormation template. A static S3 bucket name removes the intrinsic dependency between the SNS topic policy and Amazon S3.

Important: S3 bucket names must be globally unique.

The following example CloudFormation template specifies a hardcoded -Bucket-Name- value for the BucketName property:

Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
          - Sid: Statement-id
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: "SNS:Publish"
            Resource: !Ref SNSTopic
            Condition:
              ArnLike:
                aws:SourceArn: !Join
                  - ''
                  - - 'arn:aws:s3:::'
                    - '-Bucket-Name-'
      Topics:
        - !Ref SNSTopic
  S3Bucket:
    Type: AWS::S3::Bucket
    DependsOn:
      - SNSTopicPolicy
    Properties:
      AccessControl: BucketOwnerFullControl
      BucketName: "-Bucket-Name-"
      NotificationConfiguration:
        TopicConfigurations:
          - Topic: !Ref SNSTopic
            Event: s3:ObjectCreated:Put

Note: Replace -Bucket-Name- with your bucket's name. The S3Bucket resource has an explicit DependsOn attribute that's set to SNSTopicPolicy. The attribute specifies that the template creates the SNSTopicPolicy resource before the S3Bucket resource.

Use a parameter for BucketName

Parameters allow you to use the same CloudFormation template for S3 buckets with different names. During the stack creation, you can set a value for the paramBucketName parameter. Example policy:

Parameters:
  paramBucketName:
    Type: String
    Description: Bucket Name
Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
          - Sid: Statement-id
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: "SNS:Publish"
            Resource: !Ref SNSTopic
            Condition:
              ArnLike:
                aws:SourceArn: !Join
                  - ''
                  - - 'arn:aws:s3:::'
                    - !Ref paramBucketName
      Topics:
        - !Ref SNSTopic
  S3Bucket:
    Type: AWS::S3::Bucket
    DependsOn:
      - SNSTopicPolicy
    Properties:
      AccessControl: BucketOwnerFullControl
      BucketName: !Ref paramBucketName
      NotificationConfiguration:
        TopicConfigurations:
          - Topic: !Ref SNSTopic
            Event: s3:ObjectCreated:Put

Note: In the preceding example, the S3Bucket resource has an explicit DependsOn attribute that's set to SNSTopicPolicy.

Create a stack, and then update the stack

Note: In this method, the S3Bucket resource doesn't include the BucketName property. As a result, CloudFormation creates a unique bucket name for you. To avoid the circular dependency, don't use a DependsOn attribute.

First, create the stack without the NotificationConfiguration property in the S3Bucket resource. Example:

Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
          - Sid: Statement-id
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: "SNS:Publish"
            Resource: !Ref SNSTopic
            Condition:
              ArnLike:
                aws:SourceArn: !Join
                  - ''
                  - - 'arn:aws:s3:::'
                    - !Ref S3Bucket
      Topics:
        - !Ref SNSTopic
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: BucketOwnerFullControl

Add the NotificationConfiguration property in the S3Bucket resource, and then update the stack. Example:

Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
          - Sid: Statement-id
            Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: "SNS:Publish"
            Resource: !Ref SNSTopic
            Condition:
              ArnLike:
                aws:SourceArn: !Join
                  - ''
                  - - 'arn:aws:s3:::'
                    - !Ref S3Bucket
      Topics:
        - !Ref SNSTopic
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: BucketOwnerFullControl
      NotificationConfiguration:
        TopicConfigurations:
          - Topic: !Ref SNSTopic
            Event: s3:ObjectCreated:Put