Behavior of Dependent Jobs when Jobs they Depend on need Automated Retry

0

I have a Lambda that submits batch jobs for 2 categories of jobs (setup jobs, and data jobs).

The data jobs are dependent on the success of setup jobs, so I get the job ids of the setup jobs and set them as a dependency for the data files. Currently, there is no auto-retry so the data jobs fail immediately once any setup job fails.

I want to have auto-retry on the setup jobs but first want to know about the expected behavior of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

Thank you.

질문됨 일 년 전228회 조회
1개 답변
0

From the official documentation[1] we can see that:

PENDING: A job that resides in the queue and isn't yet able to run due to a dependency on another job or resource. After the dependencies are satisfied, the job is moved to RUNNABLE.

I have also replicated your scenario for your better understanding:

→ lets assume B is dependent on A, where A is the failing job

→ I created a job "A" which will fail on purpose by giving the a invalid command (slee 10 in this case), I have set the 'Job attempts' to 10

→ At the very same time, I have submitted another job "B" with the dependency "A"

→ I could see that the job "B" was in 'PENDING' status till all the retries were completed

→ Once all the retries had failed for job "A", job "B" goes into 'FAILED' status as well.

So Answering your query: Q. I want to have auto-retry on the setup jobs but first want to know about the expected behaviour of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

➢ The dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs!

Reference links:

[1] Job states - https://docs.aws.amazon.com/batch/latest/userguide/job_states.html
[2] Submitting a job - https://docs.aws.amazon.com/batch/latest/userguide/submit_job.html
[3] Job dependencies - https://docs.aws.amazon.com/batch/latest/userguide/job_dependencies.html?icmpid=docs_console_unmapped

AWS
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인