Received server error (0) from model when hosting

Question

Hi,  
  
I was trying to deploy my trained model to the endpoint and I was given a ModelError.   
 "ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (0) from model with message "Your invocation timed out while waiting for a response from container model. Review the latency metrics for each container in Amazon CloudWatch, resolve the issue, and try again.". See Link: in account 125017970330 for more information."  
  
I'm not sure what caused this issue and couldn't figure out how latency metrics in the CloudWatch would be useful in this case. Does anyone know what the approach is to solve this issue? It would also be great to know why this happens. Thanks in advance for any help!

Answer

ModelLatency is helpful because Sagemaker requires the container to respond within 60 seconds \[1]: if you see the ModelLatency at or above 60 seconds that confirms the container isn't responding fast enough. At that point you'll need to debug why your container isn't fast enough: if it's a custom container you wrote you'll need to debug it; if it's a built-in container you should reach out to AWS support for assistance.  
  
\[1] Timeout documentation https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-code-container-response

Received server error (0) from model when hosting

관련 콘텐츠