How do I update the docker image for a Batch Job Definition using CLI or API?
It looks like the APIs for RegisterJobDefinition
are "create only". You can't update a Job Definition from what I can tell from the documentation, so you can't change the reference to the Docker Image.
The JobDefinition really wants to be defined in the CDK Constructs area (or CFT) because it ties in a bunch of stuff I already have in the CDK such as databases and EFS and Secrets. That is fine, as that all that stuff is fairly static.
But Docker images are meant to change all the time, and quickly as my devs iterate code. I really don't want to specify the final Docker Image at creation time in CDK or CFT, but it looks like that's the only place to do it. I do not want to re-deploy a CDK/CFT instance just to change some code in a Docker container, that would be a slow and bad practice.
Note: a similar was asked on the old forum, but didn't really get an answer. "use :latest" isn't always the best answer for Docker version management. My devs need to be able to iterate quickly and not walk over each other.
I would like my devs to be able to change to a new Docker image and then test a batch. How can they do this quickly and easily?
Note: Here's a blog post on stable
tagging versus unique
tagging. For deployments they recommend unique. Which isn't supported with BATCH, AFAICT.
The Batch API does not provide a function to create a new revision.
I think what you are saying is one must read the JobDefinition, modify only the docker image name, and then write the result using the
registerJobDefinition()
function. This also means one should ignore drift on the CFT stack that created the JobDefinition in the first. place. Do I have that right?Yes Batch does not provide a way to update a job def (IIRC ECS does not provide one either for tasks def and what you see on the ECS console when you
create a new task rev
is just a UI facility that imports the old one and then register the new one under the same task def name with the rev at +1).I can think of a couple of ways to do this (assuming you deployed with CFN): 1) read the job def, change the image name (or likely the tag?), ignore the drift. Or 2) update the CFN stack with the new image name/tag (this will save you from reading the job def and seeing the drift but will force you to do updates via CFN).
Let's see if someone can chime in with better ideas.
(1) is ok, except some will not like 'ignore the drift'. It also could lead to inadvertent errors or races with someone updating the CFT/CDK. (2) as I mentioned in my comments is unacceptable due to very slow iteration time (and developer separation of concerns for that matter).
Consider a feature request to allow the image (or at least tag) to be overridden in the Job, just like environment variables. That would make the image late binding, which is what it should tends to be, unlike say the VPC or IAM permissions.