enable glue job queuing using AWS CLI command

0

i'm now trying to use AWS CLI to set the jobRunQueuing param after job is created, however, below is not working :

aws glue update-job --job-name my-job --job-update '{"JobRunQueuingEnabled": true}'

could you please suggest, what would be the right command to do that via CLI, tried looking at official documentation, but couldn't find a specific example of update specific param

bipinns
asked 5 days ago44 views
2 Answers
0

Hello.

Try changing the command as below.
The required parameters were "Role" and "Command".
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/glue/update-job.html

aws glue update-job --job-name test --job-update '{"JobRunQueuingEnabled": true, "Role": "arn:aws:iam::111111111111:role/glue-test", "Command":{"Name": "glueetl", "ScriptLocation": "s3://aws-glue-assets-111111111111-ap-northeast-1/scripts/test.py", "PythonVersion": "3"}}'
profile picture
EXPERT
answered 5 days ago
-1

To enable job queuing for an existing AWS Glue job using the AWS CLI, use the following command:

aws glue update-job --job-name my-job --job-update '{"JobUpdate": {"JobRunQueuingEnabled": true}}'

This command updates the specified job (my-job in this case) and sets the JobRunQueuingEnabled parameter to true. The key difference is that the JobUpdate parameter needs to be included in the JSON structure.

If you want to update multiple parameters at once, you can include them in the same JSON structure. For example:

aws glue update-job --job-name my-job --job-update '{"JobUpdate": {"JobRunQueuingEnabled": true, "Description": "Updated job with queuing enabled"}}'

Remember to replace "my-job" with the actual name of your Glue job.

After running this command, job queuing will be enabled for the specified job. This means that if the job is submitted when account quotas or limits are insufficient to start it immediately, AWS Glue will automatically queue the job and wait for resources to become available before running it.

To verify that the update was successful, you can use the get-job command to check the job's current configuration:

aws glue get-job --job-name my-job

This will return the job details, including the updated JobRunQueuingEnabled parameter.

Sources
update-job — AWS CLI 2.17.62 Command Reference
Jobs - AWS Glue

profile picture
answered 5 days ago
  • tried that but getting below error :

    Unknown options: {JobRunQueuingEnabled:true}}'

    its failing in general for other options has well : Unknown options: 4}}', {NumberOfWorkers:

    what i tried:

    aws glue update-job --job-name my-job --job-update '{"JobUpdate": {"JobRunQueuingEnabled": true}}'

    aws glue update-job --job-name my-job --job-update '{"JobUpdate": {"NumberOfWorkers": 4}}'

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions