Waiting for SageMaker CreateTrainingJob to Finish

0

I'm invoking the following task in my step function workflow:

https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-stepfunctions-tasks.SageMakerCreateTrainingJob.html

SageMakerCreateTrainingJob starts the task in an asynchronous manner, i.e. the task returns immediately after I request it to start, even if the training job may take hours to finish.

I want to have another task execute after the training job truly and completely finishes. What is an easy way to accomplish this without having to use the two techniques below?

I know there are two clunky ways to do this:

  • Using a combination of a Lambda function that calls the SageMaker API to poll for the status of a training job - i.e. to detect when it is finished - and a wait state to force a poll every X number of minutes or so.
  • Another possibility is using step function activities, but that would probably preclude me from using Lambdas as a Lambda activity worker would need to keep waiting (doing a thread wait) and constantly polling. I would also need to worry about the 15 minute Lambda time limit.
preguntada hace 3 años766 visualizaciones
1 Respuesta
0

Oops, I guess I didn't look at the integrationPattern property. Anyway, I set it to RUN_JOB and now the call is synchronous.

respondido hace 3 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas