By using AWS re:Post, you agree to the Terms of Use

Unsupported pytorch version 1.10.0 with SM Elastic Inference Accelerators

0

Hi Team,

Greetings!!

We are not able to deploy on real-time endpoint with elastic inference accelerators. Could you please have a look?

SageMaker version: 2.76.0

Code: from sagemaker.pytorch import PyTorchModel from sagemaker import get_execution_role

endpoint_name = 'ner-bert'

model = PyTorchModel(entry_point='deploy_ei.py', source_dir='code', model_data=model_data, role=get_execution_role(), framework_version='1.10.0', py_version='py38')

predictor = model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge', accelerator_type='ml.eia2.medium', endpoint_name=endpoint_name)

Error details: Unsupported pytorch version: 1.10.0. You may need to upgrade your SDK version (pip install -U sagemaker) for newer pytorch versions. Supported pytorch version(s): 1.3.1, 1.5.1, 1.3, 1.5.

Note: We are able to deploy without elastic accelerator in above code and want to use Python 3.8 version because we have some dependency libraries which supports only Python 3.8 version.

I looked at "Available DL containers" at https://github.com/aws/deep-learning-containers/blob/master/available_images.md and by looking at this section "SageMaker Framework Containers (SM support only)", SM support PyTorch 1.10.0 with Python 3.8 version.

But we would like to deploy on Elastic Inference and by looking at this section "Elastic Inference Containers" in above URL, EI containers supports only PyTorch 1.5.1 with Python 3.6. Why these containers are so outdated?

What could be the solution?

Can we specify the latest version of Python in requirements.txt file and get it installed?

Thanks, Vinayak