How can I get my AWS::ECS::Service resources out of UPDATE_IN_PROGRESS or UPDATE_ROLLBACK_IN_PROGRESS status?

Short description

Your Amazon Elastic Container Service (Amazon ECS) service can get stuck in UPDATE_IN_PROGRESS or UPDATE_ROLLBACK_IN_PROGRESS status when the service fails to launch tasks.

The following are common reasons why an Amazon ECS service fails to launch new tasks:

• Container image issues
• A lack of necessary resources to launch tasks
• A health check failure on a load balancer
• Instance configuration or Amazon ECS container agent issues

An Amazon ECS service that fails to launch tasks can cause AWS CloudFormation to get stuck in UPDATE_IN_PROGRESS status. Then, after several hours, CloudFormation rolls back to a previous configuration. If the issue that causes stack failure continues during stack rollback to a previous configuration, then the stack becomes stuck in UPDATE_ROLLBACK_IN_PROGRESS status. Finally, the stack changes to UPDATE_ROLLBACK_FAILED status.

It can take the CloudFormation stack several hours to stabilize. To stabilize your stack more quickly, complete the following steps.

Important: The following resolution is intended to help you stabilize a CloudFormation stack quickly without waiting for the stack to time out. The resolution isn't intended for production environments because the Amazon ECS service is out of sync with the known state of AWS CloudFormation. To sync resources between your Amazon ECS service and the AWS CloudFormation stack, update the stack directly.

Resolution

Change the task count of the Amazon ECS service

1. Open the Amazon ECS console.
2. Choose your cluster.
3. Select the service, and then choose Update.
4. Set Number of tasks to 0, and then save the configuration.

Identify why the Amazon ECS service can't launch new tasks

1. Open the Amazon ECS console.
2. Choose your cluster.
3. Select the service, and then choose Events.
   Note: The Events section displays the reason why your service didn't stabilize.

Resolve the issue that you found

Choose a solution based on the issue that you identified:

• Your task failed Elastic Load Balancing (ELB) health checks. For more information, see How can I get my Amazon ECS tasks that use the Amazon EC2 launch type to pass the Application Load Balancer health check?
• A container marked as essential for the task definition exited. For more information, see View Amazon ECS stopped task errors.
• You can't place a task because your container instance didn't meet the necessary requirements. For more information, see How do I resolve the "[AWS service] was unable to place a task because no container instance met all of its requirements" error in Amazon ECS?
• You receive a "cannot pull container image" error. For more information, see CannotPullContainer task errors in Amazon ECS.