Can Glue bookmarks be used between Glue workflow steps?

Question

Hello all,

I was reading this page regarding glue bookmarks:
https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html

It looks like bookmarks functionality is limited to a single Glue job processing a growing input data file incrementally over time.

A customer has a Glue workflow that has many steps (say 5 for example). They want to know if there's a way, when their Glue workflow dies on the 4th of 5 steps, to have it skip the first 3 steps on retry.

The issue they're running into is that they are hitting their DPU limit.  If the built-in Glue retries don't wait a while, then all the immediate retries will fail and the workflow crashes.

I told them Step Functions might be a better solution to catch and retry w/ exponential backoff, but they're looking to keep it simple and all in Glue, if possible.

Can anyone let me know if the Glue bookmarks and retries can be used to bookmark in between jobs in a Glue workflow, and whether Glue retries can be set to have a long retry interval with exponential backoff (like Step Functions allows)?

Accepted Answer

It seems there's nowhere we could configure the retry behavior of Glue Jobs, so your idea about using StepFunctions or Airflow to do the fine-grained flow control is better way to go.

Q: How does AWS Glue handle errors?

AWS Glue monitors job event metrics and errors, and pushes all notifications to Amazon CloudWatch. With Amazon CloudWatch, you can configure a host of actions that can be triggered based on specific notifications from AWS Glue. For example, if you get an error or a success notification from Glue, you can trigger an AWS Lambda function. Glue also provides default retry behavior that will retry all failures three times before sending out an error notification.
 from: https://aws.amazon.com/glue/faqs/

Can Glue bookmarks be used between Glue workflow steps?

Contenuto pertinente