Why do I get an error that Sagemaker Endpoint does not have multiple models

Question

Invoking a multimodel Sagemaker Endpoint, I get an error that it is not multimodel. I create it like this.

```
    create_endpoint_config_response = client.create_endpoint_config(
        EndpointConfigName=endpoint_config_name,
        ProductionVariants=[
            {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name1,
                "VariantName": model_name1,
            },
             {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name2,
                "VariantName": model_name2,
            }
        ]
    )
```

I confirm in the GUI that it in fact has multiple models. I invoke it like this:

```
    response = client.invoke_endpoint(
        EndpointName=endpoint_name, 
        TargetModel=model_name1,
        ContentType="text/x-libsvm", 
        Body=payload
    )
```

and get this error:

> ValidationError: An error occurred (ValidationError) when calling the  
> InvokeEndpoint operation: Endpoint  
> my-endpoint1 is not a multi-model endpoint  
> and does not support target model header.

The same problem was discussed  at https://github.com/aws/amazon-sagemaker-examples/issues/1026  with no resolution.  
  
How can I invoke a multimodel endpoint?

Answer

The answer is discussed here https://stackoverflow.com/questions/68802388/  
   
Two recommended improvements:  
- The error message is factually wrong. It should be amended.  
- The details  should be in the documentation https://docs.aws.amazon.com/sagemaker/latest/dg/invoke-multi-model-endpoint.html  and it would be good to further clarify whether `ModelDataUrl` is meant to be a prefix onto which `model_filename.tar.gz` is appended or whether it is a full path to a file (as appears to be the case in some Notebooks).  
  
Edited by: JoshuaFox on Aug 18, 2021 3:52 AM

Why do I get an error that Sagemaker Endpoint does not have multiple models

Relevant content