Behavior of Dependent Jobs when Jobs they Depend on need Automated Retry

0

I have a Lambda that submits batch jobs for 2 categories of jobs (setup jobs, and data jobs).

The data jobs are dependent on the success of setup jobs, so I get the job ids of the setup jobs and set them as a dependency for the data files. Currently, there is no auto-retry so the data jobs fail immediately once any setup job fails.

I want to have auto-retry on the setup jobs but first want to know about the expected behavior of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

Thank you.

asked a year ago219 views
1 Answer
0

From the official documentation[1] we can see that:

PENDING: A job that resides in the queue and isn't yet able to run due to a dependency on another job or resource. After the dependencies are satisfied, the job is moved to RUNNABLE.

I have also replicated your scenario for your better understanding:

→ lets assume B is dependent on A, where A is the failing job

→ I created a job "A" which will fail on purpose by giving the a invalid command (slee 10 in this case), I have set the 'Job attempts' to 10

→ At the very same time, I have submitted another job "B" with the dependency "A"

→ I could see that the job "B" was in 'PENDING' status till all the retries were completed

→ Once all the retries had failed for job "A", job "B" goes into 'FAILED' status as well.

So Answering your query: Q. I want to have auto-retry on the setup jobs but first want to know about the expected behaviour of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

➢ The dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs!

Reference links:

[1] Job states - https://docs.aws.amazon.com/batch/latest/userguide/job_states.html
[2] Submitting a job - https://docs.aws.amazon.com/batch/latest/userguide/submit_job.html
[3] Job dependencies - https://docs.aws.amazon.com/batch/latest/userguide/job_dependencies.html?icmpid=docs_console_unmapped

AWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions