Sagemaker Inference Endpoint Creation Failure

I'm trying to launch an inference endpoint on an ml.g4dn.2xlarge with a ~44GB model image. Every time I try to create the endpoint, it is in the creating state for 30-40 minutes and then fails with an error that says "Request to service failed. If failure persists after retry, contact customer support." There are no logs available in Cloudwatch. I get the same error for both real time and async endpoints.

I can use the same image in a Batch Transform job and the service starts up without throwing errors.

I'd love to know how to solve this issue, but I'd settle for a way to get debugging information.

Hemant Chugh
a year ago
Have you tried different type of instance or different AZ ?
Jeremy
a year ago
I have tried ml.g4dn.2xlarge and ml.g4dn.xlarge. In my case, I need to use the g4dn family of machines.
Arun Lokanatha
a year ago
how are you deploying your model. Can you share example code.
Jeremy
a year ago
I'm just using the console to create the Model, EndpointConfig, and Endpoint right now, so no code to share for those steps.

I spent time analyzing the CloudTrail events, and I can see that sagemaker goes through the process of downloading all of the image layers 4x before finally failing. None of the API calls available in CloudTrail report errors, but I think there must be a failure that's happening after the image layers have been downloaded that is triggering a series of retries. I'm stumped as to what that failure might be, since there are no events or logs associated with it.

Topics

Machine Learning & AI

Relevant content

Deploy Inference Endpoint for HG Model Pygmalion
softwarecraftmanship
asked 10 months ago
How to manage Sagemaker inference endpoint dependencies
Accepted Answer
seicaratteri
asked a year ago
SageMaker Multi Model endpoint creation fails while creating for model built on container sagemaker-scikit-learn:0.23-1-cpu-py3
AWS-User-7739087
asked 2 years ago
How do I pass images to Sagemaker inference endpoint
rePost-User-4895466
asked a year ago
Why does my Amazon SageMaker endpoint go into the failed state when I create or update an endpoint?
AWS OFFICIALUpdated a year ago
How do I troubleshoot latency with my Amazon SageMaker endpoint?
AWS OFFICIALUpdated a year ago
How can I deploy an Amazon SageMaker model to a different AWS account?
AWS OFFICIALUpdated 8 months ago
How do I resolve the "endpoint does not support the Availability Zone" error when I try to map an Amazon VPC endpoint?
AWS OFFICIALUpdated 7 months ago
Creating a DynamoDB Table with CloudFormation and Adding Items at Creation Time
EXPERT
Leeroy Hannigan
published 9 months ago
Build and Deploy Models Leveraging Cancer Gene Expression Data With SageMaker Pipelines and SageMaker Multi-Model Endpoints
EXPERT
Joshua_B
published 2 years ago