How do I resolve a ResourceInitializationError error when I try to pull an image from Amazon ECR?

Short description

When you launch an Amazon ECS task, you might receive one of the following ResourceInitializationError error messages:

"Unable to pull secrets or registry auth: The task cannot pull registry auth from Amazon ECR: There is a connection issue between the task and Amazon ECR. Check your task network configuration. RequestError: send request failed caused by: Post "https://api.ecr.region-code.amazonaws.com/": dial tcp ip.xx.xx.xx.xx:443: i/o timeout"

-or-

"unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "https://api.ecr.region-code.amazonaws.com/": dial tcp ip.xx.xx.xx.xx:443: i/o timeout. Please check your task network configuration."

These errors occur when your Amazon ECS task can't pull the container image or retrieve authentication information from Amazon ECR.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Test the task connectivity

Use the AWSSupport-TroubleshootECSTaskFailedToStart runbook to test your task connectivity. You must run the runbook in the same AWS Region where your Amazon ECS cluster resources are located. Also, use the most recently failed task ID. If the failed task is part of an Amazon ECS service, then use the most recently failed task in the service. The failed task must be visible in ECS:DescribeTasks during the automation. By default, stopped tasks are visible for 1 hour after they enter the Stopped state.

Based on the output of the automation, use one of the following manual troubleshooting steps.

Check the routes from your subnets to the internet

Check the route table of your subnet where you deployed the Amazon ECS task. Make sure that your task has a default route (0.0.0.0/0) to the internet and that the task security group allows outbound traffic on port 443.

Public subnets

If your AWS Fargate task is in a public subnet, then you must assign a public IP address to your task.

To view your task's elastic network interface and private IP address, run the following describe-tasks AWS CLI command:

aws ecs describe-tasks --cluster cluster-name --tasks task-arn

Note: Replace cluster-name with your cluster name and task-arn with your task ARN.

To check whether there's a public IP address attached to the task's network interface, run the following describe-network-interfaces command:

aws ec2 describe-network-interfaces --network-interface-ids example-eni

Note: Replace example-eni with your task's network interface.

Configure Amazon ECS to automatically assign a public IPv4 address to the network interface associated with your task. Select Turned on for Public IP when you run a new task or create a new service.

You can't automatically assign public IP addresses for existing tasks or service. Instead, to recreate the service with automatic public IP address assignment activated, run the following create-service command:

aws ecs create-service --cluster cluster-name --service-name service-name --task-definition taskdef:revision --desired-count 1 --launch-type FARGATE --network-configuration "awsvpcConfiguration={subnets=[subnet-aaaaaa,subnet-bbbbbb],securityGroups=[sg-ccccccc],assignPublicIp=ENABLED}"

Note: Replace cluster-name with your cluster name, service-name with your service name, and taskdef:revision with your task definition. Replace subnet-aaaaaa and subnet-bbbbbb with your subnets, and sg-ccccccc with your security group.

Make sure that the Amazon Virtual Private Cloud (Amazon VPC) connection for your cluster has DNS hostnames and DNS resolution activated.

Private subnets

If your task is in a private subnet, then check the following configurations based on your internet connectivity source:

• If you use a NAT gateway, then place your NAT gateway in a public subnet.
• If you use AWS PrivateLink, then verify that the VPC endpoint's security group allows inbound traffic on port 443 from the VPC CIDR or service security group.

Check your network ACL and security group settings

Make sure that your network access control list (network ACL) and security groups allow outbound access to port 443 from the subnet.

Note: Fargate tasks must have outbound access to port 443 to allow outgoing traffic and access the Amazon ECS endpoints.

(PrivateLink only) Check your Amazon VPC endpoints

If you use PrivateLink, then you must create the following required endpoints:

• For Amazon ECR, create com.amazonaws.region.ecr.dkr and com.amazonaws.region.ecr.api.
• For Amazon Simple Storage Service (Amazon S3) gateway, create com.amazonaws.region.s3.
• If you use Amazon CloudWatch to store application logs, then also create com.amazonaws.region.logs.

Note: In the preceding endpoints, replace region with your Region.

For more information, see Considerations for Amazon ECR VPC endpoints.

If your VPC doesn't have an internet gateway and your tasks use the awslogs log driver, then create a VPC endpoint for CloudWatch Logs.

Make sure that the VPC endpoint's security group allows traffic from the Fargate task security group or the VPC CIDR range on TCP port 443.

Check your IAM roles and permissions

Make sure that the AWS Identity and Access Management (IAM) task execution role has the AmazonECSTaskExecutionRolePolicy or equivalent permissions.

For more information about role permissions and the task execution role, see Amazon ECS task execution IAM role.

Related information

Verifying Amazon ECS stopped task connectivity