1 Answer
- Newest
- Most votes
- Most comments
0
It seems there's nowhere we could configure the retry behavior of Glue Jobs, so your idea about using StepFunctions or Airflow to do the fine-grained flow control is better way to go.
Q: How does AWS Glue handle errors?
AWS Glue monitors job event metrics and errors, and pushes all notifications to Amazon CloudWatch. With Amazon CloudWatch, you can configure a host of actions that can be triggered based on specific notifications from AWS Glue. For example, if you get an error or a success notification from Glue, you can trigger an AWS Lambda function. Glue also provides default retry behavior that will retry all failures three times before sending out an error notification. from: https://aws.amazon.com/glue/faqs/
answered 4 years ago
Relevant content
- asked 20 days ago
- Accepted Answerasked 5 years ago
- asked a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- How can I use a Lambda function to automatically start an AWS Glue job when a crawler run completes?AWS OFFICIALUpdated 2 years ago