Are Graviton instance types not supported for SageMaker multimodel endpoints?

Question

I am trying to create a multimodel endpoint using an ml.c7g.xlarge instance type. The error I get is: “MultiModel mode is not supported for instance type ml.c7g.xlarge.” However, according to the documentation on AWS, “Multi-model endpoints are currently supported for all CPU instances types and on single-GPU instance types.” (https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html#multi-model-endpoint-instance). Subsequently, I tried ml.c6g.xlarge and see that that also is not supported. Are Graviton instance types not supported in general? If so, why?

Answer

Hi,

Unfortunately, Graviton based instances are not supported for multi-model endpoints in Amazon SageMaker.

Multi-model endpoints currently support all the CPU instance types and on single-GPU instance types.
-> Please find the below list of Compute optimized instance types (no GPUs):
    ml.c5.large 
    ml.c5.xlarge
    ml.c5.2xlarge
    ml.c5.4xlarge
    ml.c5.9xlarge
    ml.c5.12xlarge
    ml.c5.18xlarge
    ml.c5.24xlarge
-> Supported instances for multi-model endpoints using GPU backed instances: 
[+] https://docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html#multi-model-support-gpu

Are Graviton instance types not supported for SageMaker multimodel endpoints?

Relevant content