SageMaker endpoint creation fails for Multi Model

Question

When using scikit to create multi model, it throws an exception, but when in single model it works.

Complains about model_fn implementation or ping issues, any tips on how to fix this?

e.g container={
",
    "        'Image' : image_uri,",
    "        'Mode': 'MultiModel',",
    "        'ModelDataUrl': 's3://somepatch/with/all/models/,",
    "        'Environment': {'SAGEMAKER_SUBMIT_DIRECTORY': mme_artifacts_path,",
    "                               'SAGEMAKER_PROGRAM': 'inference.py'} "

File "/miniconda3/bin/serve", line 8, in 
sys.exit(serving_entrypoint())
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/serving.py", line 144, in serving_entrypoint
start_model_server()
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/serving_mms.py", line 124, in start_model_server
modules.import_module(serving_env.module_dir, serving_env.module_name)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_modules.py", line 263, in import_module
six.reraise(_errors.ImportModuleError, _errors.ImportModuleError(e), sys.exc_info()[2])
File "/miniconda3/lib/python3.7/site-packages/six.py", line 702, in reraise
raise value.with_traceback(tb)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_modules.py", line 258, in import_module
module = importlib.import_module(name)
File "/miniconda3/lib/python3.7/importlib/__init__.py", line 118, in import_module
if name.startswith('.'):

Answer

Hi,

We would need a bit more information to help you, including a bit more detail on how you create the model (code wise) as well as more details on the errors you receive.

I would also suggest first trying out one of the available examples for multi model endpoints, like this one: [Amazon SageMaker Multi-Model Endpoints using Scikit Learn
](https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/multi_model_sklearn_home_value/sklearn_multi_model_endpoint_home_value.ipynb) and from there modify to your own needs.

SageMaker endpoint creation fails for Multi Model

Relevant content