How do I troubleshoot issues with Amazon ECS task placement constraints?

5 minute read
0

I want to use task placement constraints to run tasks on Amazon Elastic Container Service (Amazon ECS). However, the tasks get stuck in a PENDING status and fail to start.

Short description

If you configure task placement constraints on your task definition, task, or service, then you must have a container instance that matches its attributes. Otherwise, tasks get stuck in the PENDING status or fail to start, and you receive one of the following errors:

  • service (service-name) was unable to place a task because no container instance met all of its requirements. The closest matching Container instance ( a1b2c3d4-5678-90ab-cdef-11111EXAMPLE ) is missing a required attribute.
  • TaskFailedToStart: MemberOf placement constraint unsatisfied.
  • TaskFailedToStart: ATTRIBUTE

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To resolve task placement constraint issues, first check the attributes that you specified in the task placement constraint. Then, make sure that the container instances have the required attributes.

Check task placement constraints specified only in the task definition

If you use task placement constraints only in the task definition, then run the following command to check whether the container instances are missing attributes:

ecs-cli check-attributes --task-def example-task-def --cluster example-cluster --region region-code --container-instances example-container-instance-id

Note: Replace example-task-def with your task definition, example-cluster with your cluster name, region-code with your AWS Region, and example-container-instance-id with your container instance ID.

For more information, see Checking for missing attributes and debugging reason attribute errors on the GitHub website.

Check task placement constraints that are specified in the task definition, service, or task

To check the placement constraints in the task definition, complete the following steps:

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Task definitions.
  3. Choose the task definition, and then select your task definition revision.
  4. Choose Task placement.

To check the placement constraints that are specified for the service, complete the following steps:

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Clusters.
  3. On the Clusters page, choose your cluster.
  4. Under Services, choose your service.
  5. Select Configuration and networking, and then choose Task placement strategy and constraints.

To check the placement constraint that's specified on a specific task, run the following describe-tasks AWS CLI command:

    aws ecs describe-tasks  --cluster example-cluster --region region-code 
    --tasks task-id

Note: Replace example-cluster with your cluster name, region-code with your Region, and task-id with your task ID. In the command's output, check the placementConstraints value.

Make sure that your container instances have the required attributes

Use either the Amazon ECS console or the AWS CLI to check your container instance attributes.

Amazon ECS console

Complete the following steps:

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Clusters.
  3. On the Clusters page, choose your cluster.
  4. Choose the Infrastructure tab.
  5. Under Container instances, select your container instance.
  6. Choose Actions, and then select View/edit attributes.
  7. Make sure that the container instance has the required attributes specified in the placement constraint.

AWS CLI

Run the following describe-container-instances command:

aws ecs describe-container-instances --cluster example-cluster --region region-code 
--container-instances example-container-instance-id 

Note: Replace example-cluster with your cluster name, region-code with your Region, and example-container-instance-id with your container instance ID.

To view all container instances with a specific attribute, run the list-container-instances command:

aws ecs list-container-instances --cluster example-cluster --region region-code --filter "attribute:ecs.instance-type == t2.micro" 

Note: Replace example-cluster with your cluster name, region-code with your Region, and attribute:ecs.instance-type == t2.micro with your attribute. The preceding command lists all container instances in the cluster that have a t2.micro instance type.

To use the AWS CLI to add or update an attribute for a container instance, run the following put-attributes command:

aws ecs put-attributes \
    --cluster example-cluster \
    --region region-code \
    --attributes name=stack,value=production,targetId=example-container-instance-id

Note: Replace example-cluster with your cluster name, region-code with your Region, and example-container-instance-id with your container instance ID. Add your attributes to the --attributes option. The preceding example command adds the stack name and production value to the container instance.

(Optional) Update the task placement constraint

If the task placement constraints are incorrect or too restrictive, then update them in the task definition or service. Or, run the following register-task-definition to create a new task definition revision:

aws ecs register-task-definition \
    --cli-input-json file://jsonfilepath/filename.json

Note: Replace jsonfilepath.filename.json with the task definition JSON file.

To update the placement constraints on a standalone task, run the following run-task command:

aws ecs run-task \
    --cluster clustername \
    --placement-constraints exampleconstraints

Note: Replace clustername with the name of your cluster name, and exampleconstraints with your placement constraints.

AWS OFFICIAL
AWS OFFICIALUpdated a month ago