How do I troubleshoot “Internal Error” messages when stopping an AWS OpsWorks Stacks instance that's in the "stop_failed" state?

Short description

OpsWorks Stacks can return an Internal Error message when you're trying to stop an instance that's in the stop_failed state for two reasons:

• The stack's AWS Identity and Access Management (IAM) service role doesn't have the required permissions.
• The instance was stopped outside of the OpsWorks Stacks console or the OpsWorks Stacks API.
  Note: OpsWorks Stacks doesn't recognize start, stop, or restart operations performed in the Amazon Elastic Compute Cloud (Amazon EC2) console. For more information, see Manually starting, stopping, and rebooting 24/7 instances.

To resolve the error, first verify that your stack's IAM service role is configured correctly. If the issue isn't permissions related, stop the instance by running the AWS Command Line Interface (AWS CLI) stop-instance command. The command must include the --force parameter and the OpsWorks instance IDs of the instances that need to be stopped.

Resolution

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.

Verify that your stack's IAM service role is configured correctly

Follow the instructions in Allowing AWS OpsWorks Stacks to act on your behalf.

If the issue isn't permissions related, stop the instance by running the AWS CLI stop-instance command.

To stop an instance by running the AWS CLI stop-instance command

Note: You can also use AWS CloudShell to run the following commands.

1.    Return the instance IDs of the instances in your stack that are in the stopped_failed state by running the following describe-instances command:

Important: Replace your-stack-id with your stack ID. Replace your-OpsWorks-Stack-Endpoint-region with the AWS OpsWorks Stacks endpoint Region that your stack is in.

$ aws opsworks describe-instances --stack-id your-stack-id --region your-OpsWorks-Stack-Endpoint-region --query 'Instances[?Status==`stop_failed`].{Id:InstanceId}'

2.    Identify the instance IDs of the instances that you're stopping. You need these instance IDs to run the stop-instance command.

3.    (To stop a single instance) Run the following stop-instance command:

Important: Replace your-opsworks-instance-id with the instance ID that you're stopping and the your-OpsWorks-Stack-Endpoint-region with the OpsWorks Stacks endpoint Region that your stack is in.

$ aws opsworks stop-instance --instance-id your-opsworks-instance-id --force --region your-OpsWorks-Stack-Endpoint-region

-or-

(To stop multiple instances) Use custom syntax logic to run a stop-instance command that iterates through the instances that you're stopping.

Example stop-instance command that stops multiple instances

Important: Replace your-stack-id with your stack ID. Replace your-OpsWorks-Stack-Endpoint-region with the OpsWorks Stacks endpoint Region that your stack is in.

$ for Id in $(aws opsworks describe-instances --stack-id your-stack-id --region your-OpsWorks-Stack-Endpoint-region --query 'Instances[?Status==`stop_failed`].{Id:InstanceId}' --output text) ; do aws opsworks stop-instance --instance-id $Id --force --region your-OpsWorks-Stack-Endpoint-region  ; done