Unable to load trained model in Sagemaker

Question

I have trained a few models in sagemaker however I am unable to load them for prediction.

I am picking model details from: Sagemaker > Inference > Models > Container 1 section:
Image_uri = value in image
model_data = Value in model data location

then passing these values into sagemaker Model function.

When I deploy this model, it gives error: ping health check failed for AllTraffic production variant. This error doesn't come when I train a new model and deploy it.

Answer

The cause for issues like this are due to a mismatch between the base model between the training and inference endpoints. A solution similar to below would help resolve your issue.

Github repo : https://github.com/marshmellow77/sm-extend-container/blob/main/02_extend_container.ipynb. talks about how to  *extend* the existing Hugging Face DLCs by pulling them from the public ECR and running a simple Dockerfile on top of them that will install the latest available version of transformers.

Unable to load trained model in Sagemaker

Relevant content