How to create a serverless endpoint in sagemaker?
I am recreating an endpoint currently working in sagemaker for inference to a serverless endpoint. I am using one of the images ( huggingface-pytorch-inference:1.9.1-transformers4.12.3-cpu-py38-ubuntu20.04) found here -> https://github.com/aws/deep-learning-containers/blob/master/available_images.md.
everything works when i choose non serverless, i.e. provisioned option for endpoint configuration , but when i try to create one with serverless option it fails. error messages are below ( from the logs in cloudwatch) . starting with python and log4j error at the end.
'python: can't open file '/user/local/bin/deep_learning_container.py': Errno 13 permission denied. Requirement already satisfied: transformers in /opt/conda/lib/pythong3.6 ..... ..... Warning: MMS is using non-default JVM parameters: -XX: -UseContainersupport Failed to reap children process log4j: ERROR setfile(null,true) call failed. java.io.FileNotFoundException: logs/mms_log.log (No Such file or directory)
why am i getting this error ???
FYI - i have set memory to maximum allowed memory size of 6gb. for the serverless option.
The cause might be that your SageMaker Python SDK is not updated to the latest version. Please make sure you update it to the latest version as well as the AWS SDK for Python (boto3). You can use pip:
pip install --upgrade boto3
pip install --upgrade sagemakerFor a sample notebook you can have a look here. More information on the documentation page.
Relevant questions
How to define concurrency in SageMaker real-time inference
asked 3 months agoHow to create a serverless endpoint configuration?
Accepted Answerasked 3 months agoHow to configure a serverless sagemaker endpoint?
asked 3 months agoHow to create a serverless endpoint in sagemaker?
asked 3 months agoHow to grant permission to create a log file in sagemaker?
asked 3 months agoCreate endpoint from Python
Accepted Answerasked a year agoWhy I can't create a serverless sagemaker endpoint anymore ?
Accepted Answerasked a month ago[Feature Request] Serverless Inference with VPC Config
asked 4 months agoUnable to create endpoint
Accepted Answerasked 3 years agoHow to create (Serverless) SageMaker Endpoint using exiting tensorflow pb (frozen model) file?
asked 23 days ago
@loan - I'm doing this via console, not using any code or SDK at the moment.