Model error on pytorch inference container due to serialization - JSON format

Question

I am using [bring your own model](https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#bring-your-own-model) docs to deploy my trained model on aws sagemaker endpoint. I am able to parse the input and successfully ran the model on input to get the output. While passing the output in the form a json string I am getting model error. Below is the sample code for model_fn

```
def output_fn(predictions, accept='application/json'):
    probs = softmax(predictions[0], axis=1)
    outs = {
        'probability': probs.tolist()[0], 
        'confidence_score': probs.max(),
        'predicted_class': probs.argmax(),
    }   
    print(outs)
    return json.dumps(outs)
```

It's throwing the below model error for the above implementation-

> File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/botocore/client.py:1009, in BaseClient._make_api_call(self, operation_name, api_params)
   1005     error_code = error_info.get("QueryErrorCode") or error_info.get(
   1006         "Code"
   1007     )
   1008     error_class = self.exceptions.from_code(error_code)
-> 1009     raise error_class(parsed_response, operation_name)
   1010 else:
   1011     return parsed_response

I have tried to using JSONDeserialiser and content types but still facing the same issue. Any help would be highly appreciated.

Answer

Hello,

Thank you for using AWS SageMaker service and reaching out to us. I understand that you are facing Model Error

I appreciate your inquiry. However, the information and error message you have provided is not sufficient for a conclusive analysis of the issue. That being said, we always aim to assist on a best-effort basis to ensure you get the most out of SageMaker.

I would recommend probing into the following for some leads :

1. Model Signature: Please review the model's signature to understand the expected input data format, data types, and shapes.

2. Custom Predict Function

3. Input Serialization: Ensure that the serialized data sent to the SageMaker Endpoint matches what the model expects in both content and format.

For more detailed assistance, kindly reach out to  AWS Support [1] (Sagemaker), along with your issue/use case in detail and share relevant AWS resource names, and we will be happy to troubleshoot accordingly.

[1] https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-casehttps://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case****

Model error on pytorch inference container due to serialization - JSON format

Relevant content