Optimize resource for Sagemaker Pipeline

Question

Hi. I have a problem when use Sagemaker pipeline when setup pipeline processing raw data. My knowledge at each step Sagemaker will initial new instance, it spend long time for initial (by processor i have configured). So, if i want all steps on dataset only use 1 instance (may be it can difference images but in same a instance).

I am following by document: https://docs.aws.amazon.com/sagemaker/latest/dg/define-pipeline.html#define-pipeline-prereq

Answer

Hi,

Did you envision SageMaker Managed Warm Pools for training to keep your provisioned infra warm and be able to reuse it?

See https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html

Another option to explore would be Selective Execution for SageMaker Pipelines: https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-selective-ex.html

Finally, you can also cache Pipelines steps: https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-caching.html

Although not direct answer to your question about instance reuse, those capabilities will help in reducing global pipeline execution time when applicable to your use case.

Hope it helps!
Didier

Optimize resource for Sagemaker Pipeline

Relevant content