By using AWS re:Post, you agree to the Terms of Use

How to create a serverless endpoint in sagemaker?


I am recreating an endpoint currently working in sagemaker for inference to a serverless endpoint. I am using one of the images ( huggingface-pytorch-inference:1.9.1-transformers4.12.3-cpu-py38-ubuntu20.04) found here ->

everything works when i choose non serverless, i.e. provisioned option for endpoint configuration , but when i try to create one with serverless option it fails. error messages are below ( from the logs in cloudwatch) . starting with python and log4j error at the end.

'python: can't open file '/user/local/bin/': [Errno 13] permission denied. Requirement already satisfied: transformers in /opt/conda/lib/pythong3.6 ..... ..... Warning: MMS is using non-default JVM parameters: -XX: -UseContainersupport Failed to reap children process log4j: ERROR setfile(null,true) call failed. logs/mms_log.log (No Such file or directory)

why am i getting this error ???

FYI - i have set memory to maximum allowed memory size of 6gb. for the serverless option.

1 Answer

The cause might be that your SageMaker Python SDK is not updated to the latest version. Please make sure you update it to the latest version as well as the AWS SDK for Python (boto3). You can use pip:

pip install --upgrade boto3
pip install --upgrade sagemaker

For a sample notebook you can have a look here. More information on the documentation page.

answered 7 months ago
  • @loan - I'm doing this via console, not using any code or SDK at the moment.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions