Custom docker inference container failing on endpoint creation

Question

Hi, I have a docker container containing a custom inference script that keeps failing on endpoint creation with the following error:

CannotStartContainerError. Please ensure the model container for variant variant-name-1 starts correctly when invoked with 'docker run <img> serve'

The container is running fine locally. Don't know why the container keeps failing in sagemaker. Sharing the command I use to run the docker image locally for testing . 
docker run --gpus all  -v C:/Users/User/.aws:/root/.aws -p 8001:8080 e49ca31a46f5 . For local testing, I am mounting my aws credentials to the docker because the inference scripts needs to write the output to S3.

How can I debug what I am doing wrong.

Didier Durand · Answer

Hi,

It is not best practice (at all!) to embark credentials like S3 into your container image: that will create all kinds of IAM issues very probably like the one that you are currently facing.

All your credentials should be in the execution role associated with your SageMaker container instance. See https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html to implement your needed S3 credentials plus all others you need in the execution role.

Best,

Didier

Custom docker inference container failing on endpoint creation

Add your answer

Relevant content