Step Function: Concurrent job is failing if any other job fails

0

I am trying to trigger glue job via step functions by passing parameters (there will be different table names) of gule job through step function, so the issue is when one pram related glue job fails other concurrent job is stopping rather running independently

i have followed below approach to orchestrate my step function, can someone suggest

https://awstip.com/orchestrating-glue-jobs-using-aws-step-functions-to-perform-similar-etl-operations-on-multiple-d66275834162

Mohan
질문됨 7달 전270회 조회
1개 답변
0

Hello.

The behavior you're seeing (one failed job causing others to stop) typically occurs because of how your Step Functions state machine is defined. If a particular state in the state machine fails and there isn't appropriate error handling, it can cause the entire execution of the state machine to fail.

From the details you provided, it appears you are trying to run multiple Glue jobs concurrently. If you want each Glue job to run and complete independently of the others, you need to ensure your state machine is set up to handle errors appropriately.

Regards, Andrii

profile picture
전문가
답변함 7달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인