Selecting minimum instance for a Sagemaker Endpoint

Question

Good day,

I currently trying to find a way of getting the minimum instance type for deploying a Sagemaker Endpoint and is suitable for a batch job. However i have not been able to get an instance with less than 50% error rate while running performance test.

I have also explored the Sagemaker Inference Recommender, but i am getting the below error:
                           "The inference recommender job 022cc1d4-676b-11ee-9232-1609dc98c377 has failed. INVALID_INPUT : Benchmark(s) failed. To investigate the failure reason for each benchmark, please use the ListInferenceRecommendationsJobSteps API."

Answer

Hi Seun,

What type of endpoint are you deploying and what error are you getting at inference time? Not sure if you are using Batch Transform. When using Batch inference, you have a maximum payload size of GBs and maximum runtime of days. If you're using another type of deployment, such as a real-time endpoint, you might be exceeding a limit of payload size or response timeout. See [here](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html) for more details.

See [this link](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-recommender-troubleshooting.html) for a simple script to see the cause of error in the Inference Recommender job. Inference Recommender is the easiest way to assess the minimum instance type that will suit the workload.

Hope this helps.

Selecting minimum instance for a Sagemaker Endpoint

Relevant content