- Newest
- Most votes
- Most comments
Hello,
As mentioned in "Container support" and "Memory size" here https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html . The sagemaker serverless endpoint has a memory limitation of ~6GB. Whereas it supports a container of upto 10GB in size.
Kindly consider allocating upto 6GB runtime memory and select this 9GB image and you should not face any issues. In case if this does not suffice your use-case, pleas reconsider by using other suitable services that suffice your use-case or consider consulting with an AWS Solutions Architect.
Not familiar with the Segment Anything Model, however, Lambda functions (which I assume this is what you mean when you wrote AWS serverless), support an image size of up to 10 GB (this is the size of the package, i.e., the amount of code and dependencies). In addition, you can also configure it with up to 10 GB memory (this is the amount of memory your function requires to operate). Note that two values are not related to each other. You can have a small function that requires a lot of memory and a large function that requires a small amount of memory.
Relevant content
- Accepted Answerasked 3 months ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
Hi Uri,
Thanks for your response. I was talking about creating a Serverless Endpoint in Sagemaker that requires a container image of the ML model. The image size should be less than 6 GB or else it will give an error. In my case image size is 9 GB so I think we have to download specific files in an image rather than downloading it completely.
I tried with a lighter version of the ML model (5 GB in size) and didn’t receive the memory error. Below link shows the complete flow of creating a Sagemaker Endpoint:
https://drive.google.com/drive/folders/1hwZzZ9v_x7nJMRePuYSuW7VGIRIwOGpe?usp=sharing