- Newest
- Most votes
- Most comments
I am facing a similar issue. I am deploying my infra using CDK and building the image from DockerFile locally on my machine using DockerImageAsset CDK construct. I manage to deploy the stack once.After updating the container, and redeploying the stack with CDK deploy, I can see that my ECS service update fails with this error. Nothing else has changed and the container is being built on the same machine in the same way.
The error you're encountering, "CannotPullContainerError: ref pull has been retried 1 time(s): failed to unpack image on snapshotter overlayfs: mismatched image rootfs and manifest layers," is a type of CannotPullContainer error in Amazon ECS. This specific error suggests there might be an issue with the image itself or how it's being pulled.
Given that you can run the containers locally and have successfully deployed to other ECS clusters, the problem might be related to the specific cluster or its configuration. Here are a few things you can check:
-
Ensure that the task execution role associated with your Fargate tasks has the necessary permissions to pull images from your container registry (e.g., ECR).
-
Verify that the cluster has proper network connectivity to pull the image. If you're using private subnets, make sure you have a NAT gateway configured correctly.
-
Check if there are any Docker Hub rate limiting issues. If you're pulling images from Docker Hub, you might be hitting rate limits. You can authenticate Docker pulls for your container instances to increase the rate limit.
-
Examine the VPC configuration of the problematic cluster. Ensure that it has the necessary routes to access the container registry endpoint.
-
If you're using Amazon ECR, make sure your Docker client is authenticated with ECR for this specific cluster.
-
Try pulling the image manually on an EC2 instance within the same VPC as your ECS cluster to isolate whether it's a network or permissions issue.
-
Verify that the image tag in your task definition exactly matches the one in your container registry.
If none of these solutions work, you may need to investigate further by looking at the CloudWatch logs for your ECS tasks and the ECS agent logs on your container instances (if using EC2 launch type) for more detailed error messages.
Sources
CannotPullContainer task errors in Amazon ECS - Amazon Elastic Container Service
answered 2 years ago
I've updated the task on one of our other ECS's (that are deployed with the same CDK infra as code) to use the failing ECS's ECR and container. It works (I of course had update the role for the ECR so the task could pull). So none of these possible ideas seem valid.
Relevant content
asked 6 years ago
- AWS OFFICIALUpdated 7 months ago

I'm afraid this is related with the way that the image is being generated/pushed. Could you please provide the commands that you are using to build and push the image to ECR? Thanks
@Henrique that would make sense if the image having issues on ECS didn't work on another ECS deployed with the same infra as code. When I used ECR + image on the second ECS it worked without issues.
I'm also able to pull the image from the ECR and run locally without any issues.
Got it! You are using CDK for creating the infrastructure. So, I'm assuming they are exactly the same. I'm also assuming that the ECR image is the same and you are testing on both.
That said, it doesn't make much sense for me and I don't have any clue about what is going on. It should be something different between the both environment that would cause the issue. It is hard to say something without knowing your env. If you can Support contract, I'd encourage you to open a case to have this investigated. The Support team should be able to spot what is going on here looking at the envs.