How can I update my private repository credentials on an Amazon ECS container agent?

5 minute read
0

I want to update my private repository credentials on an Amazon Elastic Container Service (Amazon ECS) container agent that's supplied through AWS Secrets Manager or environment variables.

Short description

To supply private repository credentials to the Amazon ECS container agent, use either Secrets Manager in your task definition or environment variables.

Note: It's not a security best practice to store credentials in plaintext on container instances. Unless there's a direct architectural dependency on storing registry credentials within the container instance, use Secrets Manager instead.

Resolution

Update your private repository credentials with Secrets Manager

Complete the following steps:

  1. Open the Secrets Manager console.
  2. Choose your secret, and then choose Retrieve secret value.
  3. Choose Edit.
  4. Update the stored credentials for your private registry, and then choose Save.
  5. Complete the steps in the Test your updated private repository credentials section.

Update your private repository credentials with environment variables

Complete the following steps:

  1. Connect to your container instance.
  2. To review how you're supplying Docker credentials to your ECS container agent, run the following command:
    $ cat /etc/ecs/ecs.config
    This command returns the contents of the /etc/ecs/ecs.config file.

If the ECS_ENGINE_AUTH_TYPE variable is set to docker, then you're directly passing your Docker credentials by plaintext to your ECS container agent. However, it's not a best practice to use this approach. Instead, use Secrets Manager. Or, use the dockercfg format approach in the following Get a new Docker authentication value section.

If the ECS_ENGINE_AUTH_TYPE variable is set to dockercfg, then you're passing your Docker credentials by a Docker-generated authentication value that the docker login command generates. If you use this approach, then complete the steps in the Get a new Docker authentication value section.

Get a new Docker authentication value

Complete the following steps:

  1. To locally log in to your Docker account, run the following command, and then enter your new credentials:

    $ docker login
  2. To concatenate your config.json file, run the following command, and then copy the Docker-generated authentication key value:

    $ cat ~/.docker/config.json
  3. To update the ECS_ENGINE_AUTH_DATA variable, run the following command:

    $ sudo vi /etc/ecs/ecs.config
  4. In the vi editor, update the value of the ECS_ENGINE_AUTH_DATA variable to the Docker authentication key value. Example:

    ECS_CLUSTER=TestECSCluster
    ECS_ENGINE_AUTH_TYPE=dockercfg
    ECS_ENGINE_AUTH_DATA={"https://index.docker.io/v1/":{"auth":"a2vpdGhhd3M6UGFzc3dvcmQ=","email":"email@example.com"}}
  5. Complete the steps in the Restart your ECS container agent section.

Restart your ECS container agent

Complete the following steps:

  1. To restart your ECS container agent, run either of the following commands based on the Amazon Machine Images (AMIs) that your container instances are running.
    Amazon Linux 2 ECS-optimized AMIs:
    $ sudo systemctl restart ecs
    Note: Amazon Linux 1 reached its end of life on December 31, 2023. Amazon Linux AMI will no longer receive any security updates or bug fixes. For more information, see Update on Amazon Linux AMI end-of-life.
    Amazon Linux ECS-optimized AMIs:
    $ sudo stop ecs && sudo start ecs
    Note: To update the /etc/ecs/ecs.config file on other container instances in your ECS cluster, see Update your private repository credentials with environment variables. Then, follow the update process for each container instance.
  2. Complete the steps in the Test your updated private repository credentials section.

Test your updated private repository credentials

Note: In the following test example, an updated image is deployed across a cluster.

Complete the following steps:

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Clusters, and then select your cluster.
  3. Select your service, then choose Update.
  4. Select the Force new deployment check box.
  5. Choose Update.
  6. On the Deployments tab, review the new deployment. Amazon ECS gradually stops tasks under the previous deployment. Then, Amazon ECS restarts the tasks under the new deployment while ECS attempts a fresh image pull.
    Note: If you use EC2 Launch type Tasks, then your cluster must have enough resources to successfully perform a rolling update deployment type.
  7. Choose the Tasks tab, and then review each individual task and its status.

If the new task status is set to Running, then the service successfully updated the task without error.

If the new tasks last status is set to Stopped | Task failed to start, then the service updated this task, and then encountered an error. The ECS container agent can't pull a new container image and is using the old cached image. Verify that your credentials were updated, and then perform another service deployment update.

Note: To display the full details of your task and see the "pull access denied" error, choose the failed task ID. On the Description page, under Containers, select Stopped.

Example error message:

"CannotPullContainerError: Error response from daemon: pull access denied for user/reponame, repository does not exist or may require 'docker login'"

Related information

Amazon ECS container agent configuration

Authentication formats

AWS OFFICIAL
AWS OFFICIALUpdated 8 months ago