SageMaker Endpoint Debugging

Ideally JSONSerializer and JSONDeserializer should not prevent you from debugging/testing sagemaker inference endpoints locally. For an initial check, make sure that your boto3 and sagemaker libraries are up to date. Because Local Mode on sagemaker is fairly experimental, many bugs are addressed and fixed with each new version (and possibly breaking changes so keep an eye out). There are a number of examples in this aws-samples public github repository that you can refer to that run a number of combinations of frameworks and serializers. https://github.com/aws-samples/amazon-sagemaker-local-mode

This specific inference endpoint setup script utilizes CSVSerialiser/CSVDeserializer for nlp input: https://github.com/aws-samples/amazon-sagemaker-local-mode/blob/main/pytorch_nlp_script_mode_local_model_inference/pytorch_nlp_script_mode_local_model_inference.py

If you are interested in examples of the usage of JSONSerializer/JSONDeserializer then this folder should be more up your alley: https://github.com/aws-samples/amazon-sagemaker-local-mode/blob/main/huggingface_hebert_sentiment_analysis_local_serving/huggingface_hebert_sentiment_analysis_local_serving.py

If this is an issue with a specific combination that you notice do submit an issue to the public github for the sagemaker sdk (https://github.com/aws/sagemaker-python-sdk).

If the repository examples are not helpful, there is another way to speed up development. Using pre-built containers you can extend them and preinstall your required packages instead of using a requirements.txt. You can push then image to EXR and specify it in your model (example here is Pytorch):

model_instance = PyTorchModel(
    image_uri = <YourImageECRURI>,
    model_data=model_tar_path,
    role=role,
    source_dir="code",
    entry_point="inference.py",
    framework_version="1.8",
    py_version="py3"
)

This will stop the container from reinstalling your packages on every deploy. https://docs.aws.amazon.com/sagemaker/latest/dg/prebuilt-containers-extend.html

Once you are sure that your model is working as you designed, you can use the Amazon Sagemaker Inference Recommender to figure out the optimal deployment parameters for you model. This will optimize the instance type your model will run on for best performance at the lowest cost: https://docs.aws.amazon.com/sagemaker/latest/dg/inference-recommender.html