Skip to content

ECR login token expiry - reauthentication suggestions

0

We are seeing random ECR docker login token expiry errors in our Jenkins pipelines.

docker: Error response from daemon: pull access denied for my-account.dkr.ecr.us-east-1.amazonaws.com/my-project/myapp/myimage:myversion, repository does not exist or may require 'docker login': denied: Your authorization token has expired. Reauthenticate and try again.

As per AWS document, https://docs.aws.amazon.com/AmazonECR/latest/public/public-troubleshooting.html, we need to reauthenticate the ecr login. We use ecr get-login within our Jenkinsfile to run activities within a docker container as below,

sh '$(aws ecr get-login --no-include-email --region us-east-1)'
            withDockerContainer(image: 'myaccount.dkr.ecr.us-east-1.amazonaws.com/my-project/myapp/my-image:my-version'){

We were planning to update the ecr login section over a try-catch and ensure the reauthentication is done in the catch block. We would like to get a confirmation if there are any other better ways of handling this error.  

try {
                        sh '$(aws ecr get-login --no-include-email --region us-east-1)'
                    } catch (Exception e) {
                        sh 'docker logout "myaccount.dkr.ecr.us-east-1.amazonaws.com"'
                        sh '$(aws ecr get-login --no-include-email --region us-east-1)'
                    }
asked 3 years ago2.4K views
1 Answer
1
Accepted Answer

If you set up a Jenkinsfile like this, don't you log in every time?
https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html
https://www.jenkins.io/doc/book/pipeline/docker/

pipeline{
    agent any
    stages{
        stage('Hoge') {
            steps{
                script{
                    docker.withRegistry("https://${AWSAccountID}.dkr.ecr.${ECR Region}.amazonaws.com") {
                        sh("`aws ecr get-login --no-include-email --region ${ECR Region}`")
                        docker.image('my-custom-image').inside {
                          sh("echo 'hoge'")
                        }
                    }
                }
            }
        }
    }
}
EXPERT
answered 3 years ago
EXPERT
reviewed 3 years ago
  • AFAIK, this one is better than the try / catch: I don't see why the second login in cache block would succeed if initial on just failed. They will be quasi-simultanous. The issue that you described initially is more of a timeout happening long after initial login so that will be ineffective.

  • The token expiry happens quite randomly. It does not happen always. So if indeed the token has expired, we need to be doing reauthentication as per AWS suggestion. Hence, believed that the try catch will ensure that it will perform the reauthentication in case of failure.

  • Are you suggesting that changing to docker.withRegistry will solve the issue for us? May I know why the withDockerContainer gives back such issues?

  • I'm not sure why withDockerContainer would cause the token expiration issue, but I believe docker.withRegistry logs you in every time you run with a build.

  • Understood. I will give this a try. There is also a recommendation to cleanup current user docker credentials using sh 'rm ~/.docker/config.json || true' on https://docs.cloudbees.com/docs/cloudbees-ci-kb/latest/client-and-managed-controllers/withdockerregistry-step-fails-with-amazon-ecr and https://plugins.jenkins.io/amazon-ecr/. If I include that it is introducing delays in the docker image download and times out. Is the docker credentials cleanup necessary?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.