- Newest
- Most votes
- Most comments
Hello. For having managed 1000s of deployments to ECS with CFN for the past 4y, I recommend never to use the image tags, and instead always use the image digest (I usually create a docker label that uses the image tag at the time, for ease of identification).
The simple reason for that is that you want your image deployed to be immutable, if you use the tags, these can change, and if you use latest
as the default tag, then you will have a very hard time doing any form of version control of what is the version of the software actually deployed. Particularly, if you follow a pipeline like dev/stg/prod, then if you update on build the latest
image, new prod containers might be pulling the image you are testing in dev, less than optimal.
I am the maintainer of ECS Compose-X which generates the CFN templates from docker compose files during the pipeline, and I run it after the new image was built and uploaded to ECR (that way I get the sha etc., and also optionally perform an ECR vuln scan against the image).
By default, I tag the new images built in the process with the first 7 git commit characters, making it easy to link back to the code if needed, but we always, always, use the SHA for the main and sidecar containers, to ensure that we are deploying in prod the same thing that was used and tested in any prior environments.
I hope that helps :)
EDIT1: The image to use for the service/container is always a parameter to the CFN stack, as to allow for break glass solutions when you need to revert or something like that, without having to wait for a pipeline to run, which can sometimes take quite a while.
To address your concern about keeping your CloudFormation (CFN) template up-to-date with specific ECS image tags, there are a few strategies you can implement:
-
Use Parameters in your CloudFormation template: Instead of hardcoding the image URI in your template, you can use a parameter. This allows you to specify the exact image tag when you update your stack, without changing the template itself.
-
Implement image tag immutability: In Amazon ECR, you can set your repository to use immutable image tags. This prevents the "latest" tag from being overwritten and ensures that each image push creates a new tag. This practice helps maintain version consistency and prevents unintended updates.
-
Use container image digests: Instead of using image tags, you can use image digests in your task definition. Image digests are unique identifiers for a specific image version and don't change, unlike tags which can be overwritten.
-
Force new deployments: When updating your service, you can use the "forceNewDeployment" option. This ensures that even if you're using the same tag (like "latest"), ECS will pull the current image from your repository when starting new tasks.
-
Implement a CI/CD pipeline: Set up a pipeline that automatically updates your CloudFormation stack with the latest image tag whenever a new image is pushed to your ECR repository. This keeps your infrastructure code in sync with your application code.
-
Use separate templates: Consider separating your ECS task definition into a separate template from your other infrastructure. This allows you to update the task definition (including the image tag) independently of other infrastructure changes.
By implementing these strategies, you can ensure that your ECS tasks are always using the intended container image, even when you make changes to your CloudFormation template for other reasons (like adding environment variables). This approach provides better control over your deployments and helps maintain consistency between your infrastructure code and your application code.
Remember, when you update your service, ECS resolves the container image tag to a specific digest. This means that even if you update the image associated with a tag in your repository, your running tasks won't be affected until you perform a new deployment. This behavior helps ensure version consistency across your service.
Sources
Use new container image in Amazon ECS | AWS re:Post
ContainerImage - Amazon Elastic Container Service
UpdateService - Amazon Elastic Container Service
Relevant content
- Accepted Answerasked 7 months ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated 4 months ago
So, in short, the best practice is to dynamically assign the image in the CFN template. Got it!
Good feedback for using the digest, and that's what ECS does internally now. We currently use the git SHA for the image tag but the ECR image digest is a more stable identifier.