Train machine learning model using reserved instance

Question

Hi.

Is it possible to train a machine learning model with SageMaker using a reserved instance that is already up and running instead of provisioning a new instance every time which is somewhat time consuming? I'm familiar with local mode, but I understand this is not supported when using AWS SageMaker machine learning estimators.

Appreciate any suggestions for how to make the model training process in SageMaker go faster when using AWS SageMaker machine learning estimators.

Thanks,
Stefan

Accepted Answer

As of today, it's not possible to train a machine learning model with SageMaker using a reserved instance that is already up and running instead of provisioning a new instance. The service team is currently working on it, unfortunately I don't have an ETA as to when the feature will be released.

Local Mode is supported for frameworks images (TensorFlow, MXNet, Chainer, PyTorch, and Scikit-Learn) and images you supply yourself.

[Using the SageMaker Python SDK — sagemaker 2.72.3 documentation](https://sagemaker.readthedocs.io/en/stable/overview.html#local-mode)

If you want to train Built-in algorithm models simply faster, you should check the recommendation in the SageMaker document.

Example [Blazingtext-instances](https://docs.aws.amazon.com/sagemaker/latest/dg/blazingtext.html#blazingtext-instances), [Deepar-instances](https://docs.aws.amazon.com/sagemaker/latest/dg/deepar.html#deepar-instances)

If the algorithm supports it, one can also try using Pipe mode or FastFile mode. These offer some fast training job startup time. [Accelerate-model-training-using-faster-pipe-mode-on-amazon-sagemaker](https://aws.amazon.com/blogs/machine-learning/accelerate-model-training-using-faster-pipe-mode-on-amazon-sagemaker/)

Answer

ML Instance

https://aws.amazon.com/about-aws/whats-new/2021/03/amazon-ec2-inf1-instances-aws-inferentia-available-additional-regions/

Amazon EC2 Inf1 instances are available in 4 sizes, providing up to 16 Inferentia chips, 96 vCPUs, 192GB of memory, 100 Gbps of networking bandwidth and 19 Gbps of Elastic Block Store (EBS) bandwidth. These instances are purchasable On-Demand, as Reserved Instances, as Spot instances, or as part of Savings Plans and are now available in 21 regions globally,

Train machine learning model using reserved instance

Relevant content