How do I make sure that Amazon ECS uses the latest container instance image digest during deployments?

3 minute read
0

I updated my Amazon Elastic Container Service (Amazon ECS) service to run tasks with the latest image, but some tasks don't use the new image. Or, my Amazon ECS tasks fail and I receive the CannotPullContainerError error.

Short description

When you update an image in Amazon ECS, you might receive the following error message:

"CannotPullContainerError: pull image manifest has been retried 1 time(s): failed to resolve ref account-id.dkr.ecr.us-east-1.amazonaws.com/test:latest@sha256:1ed47f24455xxxxxxxxx677: account-id.dkr.ecr.us-east-1.amazonaws.com/test:latest@sha256:1ed47f24455xxxxxxxxx677: not found"

When you update an image, you must also update the service. By default, Amazon ECS resolves container image tags that are specified in the task definition to container image digests.

For a service that runs and maintains a single task, Amazon ECS uses the task to establish image digests for the task containers. For a service that runs and maintains multiple tasks, Amazon ECS uses the first task that the service scheduler started during deployment. To update the image digest when you update the image tag in the task definition, update the service and use the Force new deployment setting.

For more information about image behaviors, see Container image resolution.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Make sure that the service uses the latest image

To make sure that the service uses latest image, update the Amazon ECS service, and then use the Force new deployment option. Or, run the following AWS CLI update-service command:

aws ecs update-service --cluster clustername --service servicename --force-new-deployment 

Note: Replace clustername with your cluster name and servicename with your service name.

To use a specific image digest, update the task definition, and then modify the Image URI section:

imageUri@imagedigest

"image": "accountid.dkr.ecr.region-name.amazonaws.com/image:latest@sha256:da04638bxxxxxxxxxx,"

Resolve the "Image digest does not exist" error

If you receive an Image digest does not exist error, then check whether you deleted the image.

To check whether the image digest exists in an Amazon Elastic Container Registry (Amazon ECR) repository, run the describe-images command:

aws ecr describe-images --repository-name repositoryname --region regionname --query 'sort_by(imageDetails,& imagePushedAt)[*]'

Note: Replace repositoryname with the repository name and regionname with your AWS Region.

The command's output shows all images in the repository. If the image digest doesn't exist, then check the AWS CloudTrail event history for the BatchDeleteImage API call to determine why the image was deleted. To check whether an Amazon ECR lifecycle policy can delete an image that a running task uses, check CloudTrail for the PolicyExecutionEvent API call.

To make sure that a container image tag doesn't change, it's a best practice to turn on the tag immutability feature.

Note: The container image tag isn't resolved in services that use the CODE_DEPLOY or EXTERNAL deployment controller.

To update the image digest, complete the steps in the Make sure that the service uses the latest image section.

Deactivate version consistency

You might need to use a changeable container image tag, such as LATEST, instead of a consistent image. To configure the setting, update your container definition to set versionConsistency to disabled. By default, versionConsistency is set to enabled. When you change the value to disabled, Amazon ECS no longer resolves the container image tag to a digest. Instead, Amazon ECS uses the original image URI in the container definition for deployment.

Related information

Announcing software version consistency for Amazon ECS services

AWS OFFICIAL
AWS OFFICIALUpdated a month ago