By using AWS re:Post, you agree to the Terms of Use

Why do I get an error that Sagemaker Endpoint does not have multiple models

0

Invoking a multimodel Sagemaker Endpoint, I get an error that it is not multimodel. I create it like this.

    create_endpoint_config_response = client.create_endpoint_config(
        EndpointConfigName=endpoint_config_name,
        ProductionVariants=[
            {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name1,
                "VariantName": model_name1,
            },
             {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name2,
                "VariantName": model_name2,
            }
        ]
    )

I confirm in the GUI that it in fact has multiple models. I invoke it like this:

    response = client.invoke_endpoint(
        EndpointName=endpoint_name, 
        TargetModel=model_name1,
        ContentType="text/x-libsvm", 
        Body=payload
    )

and get this error:

ValidationError: An error occurred (ValidationError) when calling the
InvokeEndpoint operation: Endpoint
my-endpoint1 is not a multi-model endpoint
and does not support target model header.

The same problem was discussed at https://github.com/aws/amazon-sagemaker-examples/issues/1026 with no resolution.

How can I invoke a multimodel endpoint?

asked a year ago213 views
1 Answer
0

The answer is discussed here https://stackoverflow.com/questions/68802388/

Two recommended improvements:

  • The error message is factually wrong. It should be amended.
  • The details should be in the documentation https://docs.aws.amazon.com/sagemaker/latest/dg/invoke-multi-model-endpoint.html and it would be good to further clarify whether ModelDataUrl is meant to be a prefix onto which model_filename.tar.gz is appended or whether it is a full path to a file (as appears to be the case in some Notebooks).

Edited by: JoshuaFox on Aug 18, 2021 3:52 AM

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions