How can make multi model endpoint with SageMaker?

0

This is my code.

from datetime import datetime
from sagemaker.multidatamodel import MultiDataModel
mme = MultiDataModel(
    name="LV-multi-" + datetime.now().strftime("%Y-%m-%d-%H-%M-%S"),
    model_data_prefix=model_dir, # 2에서 구한 모델이 모여있는 폴더(경로)!!,
    model=sagemaker_model,  # 모델 객체 1개 우선 넣기
    sagemaker_session=sess
)

predictor = mme.deploy(
    initial_instance_count=1,
    instance_type="ml.g4dn.xlarge"
)

And error message. How can I find Ecr Image(within multi-models=true)?

ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Your Ecr Image 763104351884.dkr.ecr.ap-northeast-2.amazonaws.com/pytorch-inference:1.8.1-gpu-py3 does not contain required com.amazonaws.sagemaker.capabilities.multi-models=true Docker label(s).
1 Answer
1
Accepted Answer

Hi there - thanks for opening this thread. Multi-model endpoints are not supported on GPU instance types, see here: https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html#multi-model-endpoint-instance

In order to host a multi-model endpoint, choose a CPU instance type instead. The ECR image for CPUs will contain the required com.amazonaws.sagemaker.capabilities.multi-models=true label, see here: https://github.com/aws/deep-learning-containers/blob/master/pytorch/inference/docker/1.8/py3/Dockerfile.cpu

AWS
Heiko
answered 2 years ago
  • Thanks for your help. I struggle with this problem FOR A MONTH. X) My Best Person at this month is you!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions