Sagemaker inference on inf1 no opencv

Question

I am trying to deploy Pytorch model on ml.inf1.xlarge instance.
Image: 301217895009.dkr.ecr.us-west-2.amazonaws.com/sagemaker-neo-pytorch:1.5.1-inf-py3
My python code using some oepncv functions, and when I am trying to run the infernce I got the following error:
ModuleNotFoundError: No module named 'cv2'

I tried to add opencv-python-headless to requirements.txt, but then I got another error 
ImportError: libgthread-2.0.so.0: cannot open shared object file

How I can use opencv with the ml.inf1 instances?

Answer

When running Neo inference on Sagemaker, it’s possible to now use the Deep Learning Containers (DLC) provided by AWS. It’s also unnecessary to remain on Pytorch 1.5 when using the DLC images. In our latest released image, OpenCV is pre-installed along with a more recent version of AWS Neuron SDK. Here’s a link to the latest docker image for Pytorch 1.10 + Neuron SDK 1.17: https://github.com/aws/deep-learning-containers/blob/master/pytorch/inference/docker/1.10/py3/sdk1.17.1/Dockerfile.neuron#L83

If you’re unable to easily move to the latest Pytorch+Neuron DLC, the dockerfile link may help with resolving installation errors of OpenCV into your container.

Sagemaker inference on inf1 no opencv

Contenido relevante