By using AWS re:Post, you agree to the Terms of Use

Why do I get an error that Sagemaker Endpoint does not have multiple models


Invoking a multimodel Sagemaker Endpoint, I get an error that it is not multimodel. I create it like this.

    create_endpoint_config_response = client.create_endpoint_config(
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name1,
                "VariantName": model_name1,
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name2,
                "VariantName": model_name2,

I confirm in the GUI that it in fact has multiple models. I invoke it like this:

    response = client.invoke_endpoint(

and get this error:

ValidationError: An error occurred (ValidationError) when calling the
InvokeEndpoint operation: Endpoint
my-endpoint1 is not a multi-model endpoint
and does not support target model header.

The same problem was discussed at with no resolution.

How can I invoke a multimodel endpoint?

asked a year ago213 views
1 Answer

The answer is discussed here

Two recommended improvements:

  • The error message is factually wrong. It should be amended.
  • The details should be in the documentation and it would be good to further clarify whether ModelDataUrl is meant to be a prefix onto which model_filename.tar.gz is appended or whether it is a full path to a file (as appears to be the case in some Notebooks).

Edited by: JoshuaFox on Aug 18, 2021 3:52 AM

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions