- Newest
- Most votes
- Most comments
Job definitions (much like ECS task definitions) are versioned. This means there isn't one task/job def where you can apply standard CRUD operations but rather you are creating a new revision. When you use the job def it will use the "latest" rev by default OR you can specify a previous revision. Correct me if I am wrong but the workflow you'd use isn't tremendously different between an "update" or a "register/create" (considering that the way you will launch it won't change because it will by default always run your last rev)?
Have you found a suitable answer yet? I'm having the same problem and it would be great if there is a better way to do this.
We are using GitHub Actions for our CI/CD and I have manually created some steps that download and modify the job definition and then register the new one. It's clumsy but it works. The only problem I'm having is that we have to make sure we find and update every place that the ECR image is used every time the image is updated. That can be a royal pain, especially if you have multiple teams relying on an image.
Here's a sample of my GitHub Action workflow:
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::1112222333344455556666:role/MY_GITHUB_ACTIONS_ROLE_NAME
aws-region: us-west-2
- name: Download and modify the job definition
shell: bash
run: |
aws batch describe-job-definitions --job-definition-name MY_JOB_DEFINITION_NAME --status ACTIVE | \
jq '.jobDefinitions | unique_by(.jobDefinitionName)[] | del(
.jobDefinitionName,
.jobDefinitionArn,
.revision,
.status,
.containerOrchestrationType) |
.containerProperties.image |= gsub(".amazonaws.com/MY_IMAGE_NAME.*"; ".amazonaws.com/MY_IMAGE_NAME:${{ github.sha }}"
)' > job_definition.json
- name: Register the new job definition
shell: bash
run: |
aws batch register-job-definition \
--job-definition-name MY_JOB_DEFINITION_NAME \
--type container \
--cli-input-json file://job_definition.json
Relevant content
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
The Batch API does not provide a function to create a new revision.
I think what you are saying is one must read the JobDefinition, modify only the docker image name, and then write the result using the
registerJobDefinition()function. This also means one should ignore drift on the CFT stack that created the JobDefinition in the first. place. Do I have that right?Yes Batch does not provide a way to update a job def (IIRC ECS does not provide one either for tasks def and what you see on the ECS console when you
create a new task revis just a UI facility that imports the old one and then register the new one under the same task def name with the rev at +1).I can think of a couple of ways to do this (assuming you deployed with CFN): 1) read the job def, change the image name (or likely the tag?), ignore the drift. Or 2) update the CFN stack with the new image name/tag (this will save you from reading the job def and seeing the drift but will force you to do updates via CFN).
Let's see if someone can chime in with better ideas.
(1) is ok, except some will not like 'ignore the drift'. It also could lead to inadvertent errors or races with someone updating the CFT/CDK. (2) as I mentioned in my comments is unacceptable due to very slow iteration time (and developer separation of concerns for that matter).
Consider a feature request to allow the image (or at least tag) to be overridden in the Job, just like environment variables. That would make the image late binding, which is what it should tends to be, unlike say the VPC or IAM permissions.