Why do I get an error that Sagemaker Endpoint does not have multiple models

0

Invoking a multimodel Sagemaker Endpoint, I get an error that it is not multimodel. I create it like this.

    create_endpoint_config_response = client.create_endpoint_config(
        EndpointConfigName=endpoint_config_name,
        ProductionVariants=[
            {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name1,
                "VariantName": model_name1,
            },
             {
                "InstanceType": "ml.m5.large",
                "InitialVariantWeight": 0.5,
                "InitialInstanceCount": 1,
                "ModelName": model_name2,
                "VariantName": model_name2,
            }
        ]
    )

I confirm in the GUI that it in fact has multiple models. I invoke it like this:

    response = client.invoke_endpoint(
        EndpointName=endpoint_name, 
        TargetModel=model_name1,
        ContentType="text/x-libsvm", 
        Body=payload
    )

and get this error:

ValidationError: An error occurred (ValidationError) when calling the
InvokeEndpoint operation: Endpoint
my-endpoint1 is not a multi-model endpoint
and does not support target model header.

The same problem was discussed at https://github.com/aws/amazon-sagemaker-examples/issues/1026 with no resolution.

How can I invoke a multimodel endpoint?

已提問 3 年前檢視次數 1295 次
1 個回答
0

The answer is discussed here https://stackoverflow.com/questions/68802388/

Two recommended improvements:

  • The error message is factually wrong. It should be amended.
  • The details should be in the documentation https://docs.aws.amazon.com/sagemaker/latest/dg/invoke-multi-model-endpoint.html and it would be good to further clarify whether ModelDataUrl is meant to be a prefix onto which model_filename.tar.gz is appended or whether it is a full path to a file (as appears to be the case in some Notebooks).

Edited by: JoshuaFox on Aug 18, 2021 3:52 AM

已回答 3 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南