Recommended batch automated workflow for updating docker containers

0

How do I update the docker image for a Batch Job Definition using CLI or API?

It looks like the APIs for RegisterJobDefinition are "create only". You can't update a Job Definition from what I can tell from the documentation, so you can't change the reference to the Docker Image.

The JobDefinition really wants to be defined in the CDK Constructs area (or CFT) because it ties in a bunch of stuff I already have in the CDK such as databases and EFS and Secrets. That is fine, as that all that stuff is fairly static.

But Docker images are meant to change all the time, and quickly as my devs iterate code. I really don't want to specify the final Docker Image at creation time in CDK or CFT, but it looks like that's the only place to do it. I do not want to re-deploy a CDK/CFT instance just to change some code in a Docker container, that would be a slow and bad practice.

Note: a similar was asked on the old forum, but didn't really get an answer. "use :latest" isn't always the best answer for Docker version management. My devs need to be able to iterate quickly and not walk over each other.

I would like my devs to be able to change to a new Docker image and then test a batch. How can they do this quickly and easily?

Note: Here's a blog post on stable tagging versus unique tagging. For deployments they recommend unique. Which isn't supported with BATCH, AFAICT.

asked 2 years ago834 views
1 Answer
0

Job definitions (much like ECS task definitions) are versioned. This means there isn't one task/job def where you can apply standard CRUD operations but rather you are creating a new revision. When you use the job def it will use the "latest" rev by default OR you can specify a previous revision. Correct me if I am wrong but the workflow you'd use isn't tremendously different between an "update" or a "register/create" (considering that the way you will launch it won't change because it will by default always run your last rev)?

AWS
EXPERT
answered 2 years ago
  • The Batch API does not provide a function to create a new revision.

    I think what you are saying is one must read the JobDefinition, modify only the docker image name, and then write the result using the registerJobDefinition() function. This also means one should ignore drift on the CFT stack that created the JobDefinition in the first. place. Do I have that right?

  • Yes Batch does not provide a way to update a job def (IIRC ECS does not provide one either for tasks def and what you see on the ECS console when you create a new task rev is just a UI facility that imports the old one and then register the new one under the same task def name with the rev at +1).

  • I can think of a couple of ways to do this (assuming you deployed with CFN): 1) read the job def, change the image name (or likely the tag?), ignore the drift. Or 2) update the CFN stack with the new image name/tag (this will save you from reading the job def and seeing the drift but will force you to do updates via CFN).

    Let's see if someone can chime in with better ideas.

  • (1) is ok, except some will not like 'ignore the drift'. It also could lead to inadvertent errors or races with someone updating the CFT/CDK. (2) as I mentioned in my comments is unacceptable due to very slow iteration time (and developer separation of concerns for that matter).

    Consider a feature request to allow the image (or at least tag) to be overridden in the Job, just like environment variables. That would make the image late binding, which is what it should tends to be, unlike say the VPC or IAM permissions.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions