enable Glue job Queuing Feature from cloud formation

Question

Glue job has a new feature to use job queuing for sequencially running job run requests when max concurrency limit has reached.
This feature can be enabled by setting jobRunQueuingEnabled to true from console however, when i'm trying to create a job using cloud fromation , jobRunQueuingEnabled is being accepted.

Could anyone please suggest how to enable jobRunQueuingEnabled when creating jobs from cloud formation, or if its not yet possible then what could be an alternative option to do this while job deployment beside the manual console option, as i'm using automated pipeline to create the jobs.

re:Post Agent · Answer

While the AWS Glue job queuing feature is indeed available and can be enabled through the AWS Management Console, it appears that the CloudFormation template does not yet have a direct parameter for enabling this feature.

However, there are alternative approaches you can consider for your automated pipeline:

1. Use AWS CLI or SDK: After creating the Glue job through CloudFormation, you can use the AWS CLI or SDK to update the job and enable job queuing. This can be done by calling the UpdateJob API and setting the jobRunQueuingEnabled parameter to true.

2. Custom Resource: You can create a custom resource in your CloudFormation template that uses a Lambda function to enable job queuing after the job is created. This Lambda function would use the AWS SDK to call the UpdateJob API and set the jobRunQueuingEnabled parameter.

3. Systems Manager Automation: You could use AWS Systems Manager Automation in conjunction with your CloudFormation template to run a script that enables job queuing after the job is created.

4. CloudFormation Hook: Implement a CloudFormation hook that automatically enables job queuing for newly created Glue jobs.

As for configuring other job-related parameters in CloudFormation, you can still set various properties like MaxRetries, which specifies the maximum number of times to retry a job after a JobRun fails. You can also configure other job properties such as AllocatedCapacity, Command, Connections, DefaultArguments, and more when creating a Glue job through CloudFormation.

Remember to check the AWS documentation regularly, as CloudFormation templates are frequently updated to include new features and parameters. The ability to directly set jobRunQueuingEnabled in CloudFormation may be added in a future update.  
**Sources**  
[AWS Glue now provides job queuing - AWS](https://aws.amazon.com/about-aws/whats-new/2024/09/aws-glue-job-queuing/)  
[AWS::Glue::Job - AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-job.html)

Riku_Kobayashi · Answer

Hello.

I answered the question below, but if you want to change it using AWS CLI, try changing the command as follows.  
https://repost.aws/ja/questions/QUElUIgnGXSo69swguAys_dA/enable-glue-job-queuing-using-aws-cli-command  
```
aws glue update-job --job-name test --job-update '{"JobRunQueuingEnabled": true, "Role": "arn:aws:iam::111111111111:role/glue-test", "Command":{"Name": "glueetl", "ScriptLocation": "s3://aws-glue-assets-111111111111-ap-northeast-1/scripts/test.py", "PythonVersion": "3"}}'
```

enable Glue job Queuing Feature from cloud formation

Relevant content