How do I resolve the error "Failed to receive X resource signal(s) within the specified duration" for EC2 Windows instances in AWS CloudFormation?

Short description

You get this error when CloudFormation doesn't receive success signals for resources that have a CreationPolicy attribute specified with a ResourceSignal in it. The error might occur for an Amazon EC2 instance, Auto Scaling group, or a wait condition.

Note: The following resolution applies only to CloudFormation stacks that you create with Windows instances. For Linux instances, see How do I resolve the error "Failed to receive X resource signal(s) within the specified duration" in AWS CloudFormation?

Resolution

Based on your use case, use the following troubleshooting steps to resolve your issue.

Note: To prevent a stack rollback, choose Preserve successfully provisioned resources for Stack failure options in the CloudFormation console. When you choose this option, the failed instance isn't terminated until you delete the stack.

The cfn-bootstrap MSI package isn't installed on one or more instances of the AWS CloudFormation stack

To confirm that the cfn-signal script is installed on the instance that's configured to send signals to the stack, complete the following steps:

1. Use RDP to connect to your Windows instance.
2. Confirm that the cfn helper scripts package is installed. Run the following command in Windows PowerShell:

   Get-Package -name aws-cfn-bootstrap

   Important: By default, CloudFormation helper scripts are installed on Amazon Windows Amazon Machine Images (AMIs). To install the helper scripts, see CloudFormation helper scripts reference.

The AWS CloudFormation template has syntax errors or incorrect values

To find the errors and incorrect values, complete the following steps:

1. In a code editor, open the CloudFormation template for your stack. Then, find the UserData property section.
2. Check for errors with syntax, missing spaces, misspellings, and other typos.
3. Confirm that the values for the stack, resource, and AWS Region properties are correct.
   Note: Check for syntax errors or incorrect values in the bootstrap script that's included in the UserData property. The script calls the cfn-signal.

If you signal within the cfn-init commands, then look for information about the signal in the cfn-init logs. To search for errors in the cloud-init or cfn-init logs, use RDP to connect to your instance. Then, use the keyword "error" or "failure" to search for detailed error or failure messages in the following logs:

C:\cfn\log\cfn-init.log  
C:\cfn\log\cfn-init-cmd.log  
C:\cfn\log\cfn-wire.log

The timeout property for the CreationPolicy attribute is very low

The timeout property's value is defined by the CreationPolicy attribute. Confirm that the value is high enough to run tasks before the cfn-signal script sends signals to CloudFormation resources.

To check the timeout property value and compare the signaling and resource failure timestamps, complete the following steps:

1. In a code editor, open the CloudFormation template for your stack to find the timeout property value.
   Note: The timeout property value is the maximum amount of time that CloudFormation waits for a signal before it returns an error.

2. To get an estimate of when the cfn-signal script is activated, use RDP to connect to the instance. Then, run the following command:

   C:\cfn\log\cfn-init.log

   Compare the start and end timestamps that are logged in the file to get an estimate of the time it took to bootstrap. Modify the timeout value as needed. The maximum time that you can specify is 12 hours.

   The log file shows a timestamp when the SUCCESS signal is sent to CloudFormation resources.
   Example:

   2019-01-11 12:46:40,101 [DEBUG] Signaling resource EC2Instance in stack XXXX with unique ID i-045a536a3dfc8ccad and status SUCCESS

3. Open the CloudFormation console.

4. Choose the Events view.

5. Choose Status reason. Expand the row for the event with the status reason "Failed to receive X resource signal(s) within the specified duration."

6. Compare the signaling timestamp with the resource failure timestamp.
   Note: For a successful completion, the script must send the signal before the instance is created or fails to create.

The cfn-signal isn't sent from the Amazon EC2 instance

Verify that the signal that CloudFormation received came from the instance. Check the cfn wire log that's available at C:\cfn\log\cfn-wire.log. If the response isn't 200, then there might be a connectivity issue between your instance and the CloudFormation's endpoint.

If you configure a reboot in your instance and the cfn-signal is set in the UserData section, then the signal might not be sent. This is because UserData runs only once. For more information, see How do I stop my Amazon EC2 Windows instance from signaling back as CREATE_COMPLETE before the instance finishes bootstrapping?

When you send signals from somewhere that's not your instance, use the SignalResource API. For example, you can use an AWS Lambda function to call the SignalResource API, and then send the signal to the stack. If you get an error, then use CloudWatch Logs to check your Lambda logs to understand why the signal wasn't sent to the stack.