Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

ModelError while using InvokeEndpoint with Sagemaker

0

I've deployed a pytorch model onto Sagemaker and have confirmed that the endpoint is in service, but using the invoke endpoint operation results in the following error: "ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from primary and could not load the entire response body." I've ensured that the tar.gz file contains the inference code, model, tokenizer, etc., and have tried simplifying the model script so it returns the bare minimum to make sure there are no problems with how data is being processed, but the error persists. If anyone has experienced this before or has any advice on what to try, I'd appreciate any help!

Topics
Machine Learning & AIStorage
Tags
Amazon SageMaker
Language
English
asked 3 years ago1.8K views
1 Answer
0
Accepted Answer

Hi Siddharth,

There's not a whole lot of information for us to go off from here. It's a general error, but I've faced it before when passing the inputs to the model incorrectly. What type of data are you trying pass through?

A suggestion is to try run Sagemaker in local mode. If you run locally, do you experience a similar error?

To help more, I'd need to see any further tracebacks, cloudwatch logs or more information on what you're trying to achieve.

Let me know how you get on.

Cheers James

answered 3 years ago
  • rePost-User-3781855
    3 years ago

    Hi James, I'm trying to pass in a json, and I've used json.dumps during my InvokeEndpoint calls as well as json.loads within the inference code when processing input data. I haven't tried local mode, but I'll certainly give it a try!

    I think it's worth noting that I haven't used a pytorch container, and instead have only defined dependencies that my code runs upon. Would this cause issues? If so, would it be possible to point me towards some sort of documentation on creating a dockerfile/image/pytorch container since I've been struggling to learn about it online. Thanks so much.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content