How do I resolve "CannotPullContainerError" errors when I launch an EC2 task in Amazon ECS?

Short description

The CannotPullContainerError occurs for the following reasons:

• An Amazon EC2 launch type task can't pull the image because you incorrectly configured the network.
• An AWS Identity and Access Management (IAM) role doesn't have permissions to pull the image.
• You reached your DockerHub rate quota.
• The image name or tag doesn't exist.
• The task definition references an image that's in a different AWS account.

Resolution

Use the AWSSupport-TroubleshootECSTaskFailedToStart runbook to troubleshoot

To troubleshoot Amazon ECS tasks that fail to start, use the AWSSupport-TroubleshootECSTaskFailedToStart runbook. When you run the runbook, you must use the most recently failed task ID. If the failed task is part of an Amazon ECS service, then use the most recently failed task in the service. The failed task must be visible in ECS:DescribeTasks during the automation. By default, Amazon ECS tasks are visible for 1 hour after they enter the Stopped state.

To initiate the runbook, see AWSSupport-TroubleshootECSTaskFailedToStart. Based on the output of the automation, use one of the following resolutions.

Important: Use the AWSSupport-TroubleshootECSTaskFailedToStart runbook in the same AWS Region where your Amazon ECS cluster resources are located.

Your Amazon ECS container instance network configuration is incorrect

If your Amazon ECS container instance doesn't have an internet connection, then the instance can't reach the Amazon Elastic Container Registry (Amazon ECR) endpoint. Without a connection to the endpoint, the instance can't pull the image. To verify that your Amazon ECS container instance has internet access, take the following actions:

• Confirm that your instances have access to the internet through either an internet gateway or NAT gateway. To troubleshoot NAT gateway connectivity issues, see Why can't my Amazon EC2 instance in a private subnet connect to the internet using a NAT gateway?
  Note: You can use AWS PrivateLink instead of a NAT gateway. To avoid errors, make sure to correctly configure AWS PrivateLink.
• Verify that inbound HTTPS access is allowed through port 443 at the instance, security group, and network access control list (network ACL) levels. To troubleshoot internet gateway connectivity issues, see Why can't my Amazon EC2 instance access the internet through an internet gateway?
• If you use a virtual private cloud (VPC) endpoint to connect to an Amazon ECR endpoint, then check your Amazon ECS container instance security groups. Allow your Amazon ECS container instance to send traffic to security groups for your VPC endpoint.
• If you receive the error message CannotPullContainerError: API error, then see How do I resolve Amazon ECS tasks that fail to pull images from Amazon ECR?

Your IAM role doesn't have permissions to pull images

Confirm that the instance IAM role that's associated with the instance profile has permission to access the Amazon ECR repository.

Note: The AmazonEC2ContainerRegistryReadOnly AWS managed policy provides the minimum required permissions to pull images.

The DockerHub pull rate limit is reached

If you're trying to pull an image from DockerHub and you reached your pull rate quota, then you receive the following error message:

"CannotPullContainerError: inspect image has been retried 5 time(s): httpReaderSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/manifests/sha256:2bb501e6429 Too Many Requests - Server message: toomanyrequests:"

To resolve this issue, see How do I resolve the error "CannotPullContainerError: You have reached your pull rate limit" in Amazon ECS?

The image name or tag doesn't exist

If you define an image or tag that's hosted in Amazon ECR but the image or tag doesn't exist, then you receive the following error message:

"Cannotpullcontainererror: pull image manifest has been retried 1 time(s): failed to resolve ref 123456789.dkr.ecr.ap-southeast-2.amazonaws.com/image-name:tag: 123456789**.dkr.ecr.ap-southeast-2.amazonaws.com/image-name:tag: not found**"

To resolve this issue, verify that the image field has the correct value in your task definition. Confirm that the corresponding repository and tag exists in the container registry that you're pulling from.

If your task pulls an image from a registry other than Amazon ECR and your task is missing information, then you receive a different error. The following error occurs when the image doesn't exist, the tag doesn't exist, or you didn't provide the registry credentials:

"Cannotpullcontainererror: pull image manifest has been retried 1 time(s): failed to resolve ref docker.io/library/invalid-name:non-existenttag: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed"

To resolve this issue, check the image field in your task definition. Confirm that the image exists in the corresponding container registry. If the image exists, then make sure that you provide the correct credentials. For more information, see Using non-AWS container images in Amazon ECS.

The image that's referenced in the task definition is in a different account

If your image is in a different account, then you might receive the following error message:

"Cannotpullcontainererror: pull access denied for 123456789***.dkr.ecr.us-east-1.amazonaws.com/image-name:tag, repository does not exist or may require 'docker login': denied: User: arn:aws:sts::012345678**:assumed-role/role-name/ec2 instance id is not authorized to perform: ecr:BatchGetImage on resource: arn:aws:ecr:us-east-1:123456789*:repository/image-name:tag because no resource-based policy allows the ecr:BatchGetImage action"

To resolve this issue, you must allow the ecr:batchGetImage action in the repository policies in the account that the image is located in.

For example, the instance that's associated with your Amazon ECS cluster is in Account A and the task definition references an image in Account B. The repository policies in Account B must allow ecr:batchGetImage for the instance IAM role in Account A. You must also activate cross-account permission in the repository policy. Then, confirm that the instance in Account A can pull the image from Account B. For more information, see How do I let a secondary account push or pull images in my Amazon ECR image repository?