FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/code' when deploying SageMaker Multi-model Endpoint (MME)

Question

![Enter image description here](/media/postImages/original/IMiXyVYzfRQp6i7mAmRVTbfQ)

The figure above illustrates the directory tree for model_data model.tar.gz, where inference.py has dependency on src/serving. I made sure folder "code" is in the root directory of the model artifact tar.gz so that it will be unzipped into opt/ml/model. I can post what codes are inside inference.py if needed but I doubt that's the reason of error. The full traceback is posted in the bottom. Thanks in advance.

```
from sagemaker.multidatamodel import MultiDataModel
from sagemaker.xgboost.model import XGBoostModel

model_data_prefix = f"s3://{write_bucket}/{write_prefix}/MME/house-price-model"
model_data = f"{model_data_prefix}/model.tar.gz"

xgboost_model = XGBoostModel(
    model_data=model_data,
    role=sagemaker_role,
    vpc_config=vpc_config,
    sagemaker_session=sess,
    enable_network_isolation=True,
    entry_point="inference.py",
    framework_version="1.7-1",
)

mme = MultiDataModel(
    name='house-price-model-mme',
    model_data_prefix=model_data_prefix,
    model=xgboost_model, 
    sagemaker_session=sess,
)
print(list(mme.list_models()))

predictor = mme.deploy(
    initial_instance_count=1,
    instance_type="ml.c5.xlarge",
)
```

```
Traceback (most recent call last):

File "/miniconda3/bin/serve", line 8, in

sys.exit(serving_entrypoint())

File "/miniconda3/lib/python3.9/site-packages/sagemaker_xgboost_container/serving.py", line 167, in serving_entrypoint

start_mxnet_model_server()

File "/miniconda3/lib/python3.9/site-packages/sagemaker_xgboost_container/serving_mms.py", line 149, in start_mxnet_model_server

modules.import_module(serving_env.module_dir, serving_env.module_name)

File "/miniconda3/lib/python3.9/site-packages/sagemaker_containers/_modules.py", line 253, in import_module

_files.download_and_extract(uri, _env.code_dir)

File "/miniconda3/lib/python3.9/site-packages/sagemaker_containers/_files.py", line 140, in download_and_extract

elif tarfile.is_tarfile(uri):

File "/miniconda3/lib/python3.9/tarfile.py", line 2802, in is_tarfile

t = open(name)

File "/miniconda3/lib/python3.9/tarfile.py", line 1821, in open

return func(name, "r", fileobj, **kwargs)

File "/miniconda3/lib/python3.9/tarfile.py", line 1885, in gzopen

fileobj = GzipFile(name, mode + "b", compresslevel, fileobj)

File "/miniconda3/lib/python3.9/gzip.py", line 173, in __init__

fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')

FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/code'
```

Thayin_M · Answer

~~Testing with the below, I can get this to work. Key here is that you want to obtain the repacked model when creating the MultiModel object as this will contain both the contents of `model_data` and `source_dir`. I did notice that there are some issues within `service.py` which should be addressed before model can be deployed to an Endpoint.~~

I have updated the below code to set `enable_network_isolation` to `False` so that a `source_dir.tar.gz` can be generated that is independent of our `model.tar.gz` since `model.tar.gz` will not be unpacked during startup for MultiModel Endpoints. I did notice that there are some issues within `service.py` which should be addressed before model can be deployed to an Endpoint.

All testing was done using sagemaker version 2.240.0

>Traceback (most recent call last):
File "/miniconda3/lib/python3.9/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper
return fn(*args, **kwargs)
   File "/opt/ml/code/inference.py", line 9, in model_fn
     return serve.model_loading(model_dir, 'classification')
   File "/opt/ml/code/shared/serving.py", line 57, in model_loading
     model_prefix = mc.CLASSIFICATION_OUTCOME
 NameError: name 'mc' is not defined

```python
import sagemaker
from sagemaker.multidatamodel import MultiDataModel
from sagemaker.xgboost.model import XGBoostModel

session = sagemaker.Session()
write_bucket = session.default_bucket()
write_prefix = "sagemaker_mme_bug/MME/house-price-model"
model_data_prefix = f"s3://{write_bucket}/{write_prefix}/"
role = sagemaker.get_execution_role()

xgboost_model = XGBoostModel(
    name="xgboost-model",
    role=role,
    model_data=model_data_prefix,
    # vpc_config=vpc_config,
    sagemaker_session=session,
    enable_network_isolation=False,  # Needs to be set to False to decouple code from model.
    entry_point="inference.py",
    framework_version="1.7-1",
    source_dir="./code",  # Upload to a S3 bucket
)

session.upload_data(path='model.tar.gz', bucket=write_bucket, key_prefix=write_prefix)

mme = MultiDataModel(
    name='house-price-model-mme',
    model_data_prefix=model_data_prefix,
    sagemaker_session=session,
    model=xgboost_model
)
print(list(mme.list_models()))

predictor = mme.deploy(
    initial_instance_count=1,
    instance_type="ml.c5.xlarge",
)
```

rwxb · Answer

Your folder structure looks to be correct. Please share  the whole  repo so that I replicate this

FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/code' when deploying SageMaker Multi-model Endpoint (MME)

Relevant content