Waiting for SageMaker CreateTrainingJob to Finish

0

I'm invoking the following task in my step function workflow:

https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-stepfunctions-tasks.SageMakerCreateTrainingJob.html

SageMakerCreateTrainingJob starts the task in an asynchronous manner, i.e. the task returns immediately after I request it to start, even if the training job may take hours to finish.

I want to have another task execute after the training job truly and completely finishes. What is an easy way to accomplish this without having to use the two techniques below?

I know there are two clunky ways to do this:

  • Using a combination of a Lambda function that calls the SageMaker API to poll for the status of a training job - i.e. to detect when it is finished - and a wait state to force a poll every X number of minutes or so.
  • Another possibility is using step function activities, but that would probably preclude me from using Lambdas as a Lambda activity worker would need to keep waiting (doing a thread wait) and constantly polling. I would also need to worry about the 15 minute Lambda time limit.
gefragt vor 3 Jahren766 Aufrufe
1 Antwort
0

Oops, I guess I didn't look at the integrationPattern property. Anyway, I set it to RUN_JOB and now the call is synchronous.

beantwortet vor 3 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen