Step Function: Concurrent job is failing if any other job fails

0

I am trying to trigger glue job via step functions by passing parameters (there will be different table names) of gule job through step function, so the issue is when one pram related glue job fails other concurrent job is stopping rather running independently

i have followed below approach to orchestrate my step function, can someone suggest

https://awstip.com/orchestrating-glue-jobs-using-aws-step-functions-to-perform-similar-etl-operations-on-multiple-d66275834162

Mohan
質問済み 7ヶ月前270ビュー
1回答
0

Hello.

The behavior you're seeing (one failed job causing others to stop) typically occurs because of how your Step Functions state machine is defined. If a particular state in the state machine fails and there isn't appropriate error handling, it can cause the entire execution of the state machine to fail.

From the details you provided, it appears you are trying to run multiple Glue jobs concurrently. If you want each Glue job to run and complete independently of the others, you need to ensure your state machine is set up to handle errors appropriately.

Regards, Andrii

profile picture
エキスパート
回答済み 7ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ