How do I troubleshoot a State Manager association that's stuck in "Failed" or "Pending" status?

Resolution

Verify your configurations

Roles and permissions for Systems Manager

Check that you attached the AmazonSSMFullAccess AWS managed policy to the user that must create an association. Be sure to use an AWS Identity and Access Management (IAM) role with permissions to retrieve and run Systems Manager documents. For the minimum required permissions for the role, see AmazonSSMManagedInstanceCore.

Run Command

If you want to use Run Command, a capability of AWS Systems Manager, then verify that the target instance is a managed instance. For more information, see Working with managed nodes.

Automation Association

If State Manager targets the automation document, then verify that you have the necessary permissions to run the automation. For more information, see Configuring a service role (assume role) access for automations.

SSM Agent

Make sure that you installed the latest version of AWS Systems Manager Agent (SSM Agent) on your instance. For more information, see Working with SSM Agent.

Connectivity

Verify that you configured the following resources and settings:

• You can access the instance metadata for all your target instances, except for on-premises managed instances.
• The target instance has outbound internet access on TCP port 443 to the ec2messages.region-id.amazonaws.com and ssm.region-id.amazonaws.com Systems Manager regional service endpoints.

Troubleshoot an association that's stuck in Pending or Failed status

If the association’s status is Pending or Failed after you configured Systems Manager and its capabilities correctly, then complete the following steps:

1. Open the Systems Manager console.
2. In the navigation pane, choose State Manager.
3. Choose the Association Id for the association that's stuck in the Pending or Failed state.
4. Choose the Execution history tab to view the invocation history. If the history lists invocations, then choose Execution id to see the resource type, status, and other details.
   Note: If there aren't any invocations listed in the history, then verify that your instance is a managed instance. On the Systems Manager console, verify that your instance appears under Managed instances, and the SSM Agent ping status is Online.
5. Choose Resource ID, and then select the target instance Execution ID Association execution targets.
6. Select the target instance Resource id, then choose Output.

The output displays details and an error message about why the association failed. For more information on error messages, see the following troubleshooting guides:

• For Automation, a capability of AWS Systems Manager, see Troubleshooting Systems Manager Automation.
• For Run Command, see Troubleshooting Systems Manager Run Command.

If your instance doesn't appear under Managed instances, or if the SSM Agent ping status is Connection lost, then troubleshoot the issue. For instructions, see Why isn't Systems Manager showing my Amazon EC2 instance as a managed instance?

Note: The output differs depending on the Systems Manager document that you use. For more information, see AWS Systems Manager documents.

Review SSM Agent logs

For more details about Run Command document failure, review your SSM Agent logs in the following directories:

For Windows:

• %PROGRAMDATA%\Amazon\SSM\Logs\amazon-ssm-agent.log
• %PROGRAMDATA%\Amazon\SSM\Logs\errors.log
• %PROGRAMDATA%\Amazon\SSM\Logs\audits\amazon-ssm-agent-audit-YYYY-MM-DD
  Note: Replace YYYY-MM-DD with the date that corresponds with your document failure.

For Linux and macOS:

• /var/log/amazon/ssm/amazon-ssm-agent.log
• /var/log/amazon/ssm/errors.log
• /var/log/amazon/ssm/audits/amazon-ssm-agent-audit-YYYY-MM-DD
  Note: Replace YYYY-MM-DD with the date that corresponds with your document failure.

Note: SSM Agent writes stderr and stdout to the /var/lib/amazon/ssm directory.

Related information

Learn about statuses returned by Systems Manager Automation