a glue job in a step function is taking so long to continue the next step

Hello!

This is an expected behavior in case of .sync integration with AWS Glue. Service integrations that use the .sync pattern require additional IAM permissions where Step Functions will make use of a managed Eventbridge rule to monitor the status of the job. However, AWS Glue does not support Eventbridge integration and thus, Step Functions polls the job status using the GetJobRun API call to fetch the status of the job.

By design, the polling schedule is once per minute for the first 10 minutes, and every 5 minutes thereafter. Hence, although your Glue job may complete in just a few seconds, Step Functions will place the GetJobRun call to fetch the job status only after 1 minute, post which the state transition occurs based on the status.

A possible workaround in order to reduce this state transition delay would be to have a request response integration with Glue and use a job poller to manually place the GetJobRun call in the workflow to get the status of the job much quickly.

If your workflow does not have a .sync integration or if there are any other queries specific to your setup, we would require details that are non-public information. Please open a support case with AWS using the following link.