Behavior of Dependent Jobs when Jobs they Depend on need Automated Retry

0

I have a Lambda that submits batch jobs for 2 categories of jobs (setup jobs, and data jobs).

The data jobs are dependent on the success of setup jobs, so I get the job ids of the setup jobs and set them as a dependency for the data files. Currently, there is no auto-retry so the data jobs fail immediately once any setup job fails.

I want to have auto-retry on the setup jobs but first want to know about the expected behavior of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

Thank you.

gefragt vor einem Jahr228 Aufrufe
1 Antwort
0

From the official documentation[1] we can see that:

PENDING: A job that resides in the queue and isn't yet able to run due to a dependency on another job or resource. After the dependencies are satisfied, the job is moved to RUNNABLE.

I have also replicated your scenario for your better understanding:

→ lets assume B is dependent on A, where A is the failing job

→ I created a job "A" which will fail on purpose by giving the a invalid command (slee 10 in this case), I have set the 'Job attempts' to 10

→ At the very same time, I have submitted another job "B" with the dependency "A"

→ I could see that the job "B" was in 'PENDING' status till all the retries were completed

→ Once all the retries had failed for job "A", job "B" goes into 'FAILED' status as well.

So Answering your query: Q. I want to have auto-retry on the setup jobs but first want to know about the expected behaviour of the dependent data jobs -- If I put the data jobs in queue with a dependency on the setup jobs, will the data jobs fail at the FIRST failure? Or will the dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs?

➢ The dependent data jobs stay in 'PENDING' state until all retry attempts are made on the setup jobs!

Reference links:

[1] Job states - https://docs.aws.amazon.com/batch/latest/userguide/job_states.html
[2] Submitting a job - https://docs.aws.amazon.com/batch/latest/userguide/submit_job.html
[3] Job dependencies - https://docs.aws.amazon.com/batch/latest/userguide/job_dependencies.html?icmpid=docs_console_unmapped

AWS
beantwortet vor einem Jahr

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen

Relevanter Inhalt