AWS SageMaker - Extending Pre-built Container, Deploy Endpoint Failed. No such file or directory: 'serve'"

Question

I am trying to deploy the SageMaker Inference Endpoint by extending the Pre-built image. However, it failed with "FileNotFoundError: [Errno 2] No such file or directory: 'serve'"

My Dockerfile

```
ARG REGION=us-west-2

# SageMaker PyTorch image
FROM 763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-inference:1.12.1-gpu-py38-cu116-ubuntu20.04-ec2

RUN apt-get update

ENV PATH="/opt/ml/code:${PATH}"

# this environment variable is used by the SageMaker PyTorch container to determine our user code directory.
ENV SAGEMAKER_SUBMIT_DIRECTORY /opt/ml/code

# /opt/ml and all subdirectories are utilized by SageMaker, use the /code subdirectory to store your user code.
COPY inference.py /opt/ml/code/inference.py

# Defines inference.py as script entrypoint 
ENV SAGEMAKER_PROGRAM inference.py
```

CloudWatch Log From /aws/sagemaker/Endpoints/mytestEndpoint
```
2022-09-30T04:47:09.178-07:00
Traceback (most recent call last):
  File "/usr/local/bin/dockerd-entrypoint.py", line 20, in 
    subprocess.check_call(shlex.split(' '.join(sys.argv[1:])))
  File "/opt/conda/lib/python3.8/subprocess.py", line 359, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/opt/conda/lib/python3.8/subprocess.py", line 340, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/opt/conda/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/conda/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
Traceback (most recent call last): File "/usr/local/bin/dockerd-entrypoint.py", line 20, in  subprocess.check_call(shlex.split(' '.join(sys.argv[1:]))) File "/opt/conda/lib/python3.8/subprocess.py", line 359, in check_call retcode = call(*popenargs, **kwargs) File "/opt/conda/lib/python3.8/subprocess.py", line 340, in call with Popen(*popenargs, **kwargs) as p: File "/opt/conda/lib/python3.8/subprocess.py", line 858, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/opt/conda/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename)

2022-09-30T04:47:13.409-07:00
FileNotFoundError: [Errno 2] No such file or directory: 'serve'
```

holopekochan · Accepted Answer

Should use the Sagemaker image 
```
763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-inference:1.12.1-gpu-py38-cu116-ubuntu20.04-sagemaker
```
instead of ec2

```
763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-inference:1.12.1-gpu-py38-cu116-ubuntu20.04-ec2
```

Ivan · Answer

Hi, @holopekochan!

The `serve` script is installed by [SageMaker PyTorch Inference Toolkit](https://github.com/aws/sagemaker-pytorch-inference-toolkit/blob/0513c7a83d95dd0124002102d4b1c4f8c603e406/setup.py#L64) when you pip-install it in the [Dockerfile](https://github.com/aws/deep-learning-containers/blob/07646a959393e8622c1b3e9704dba58c0f7fed73/pytorch/inference/docker/1.12/py3/cu116/Dockerfile.gpu#L279).

However, it's hard to say why it's not found in your container. Are you sure that you use the inference container, not training container, for your endpoint? If you go to the *AWS Console > Amazon SageMaker > Models > your model*, what ECR image it shows in *Container 1 - Image*?

It will be useful if you can share the code that you used to setup the SageMaker PyTorch estimator (if any) how you define your PyTorchModel and how you deploy() it.

AWS SageMaker - Extending Pre-built Container, Deploy Endpoint Failed. No such file or directory: 'serve'"

Relevant content