How to check/determine image/container size for aws managed images ?

1

I'm using one of the images listed here https://github.com/aws/deep-learning-containers/blob/master/available_images.md, to create an model such that I can tie that up with a sagemaker serverless endpoint , but I keep getting "failed reason: Image size 15136109518 is greater that suppported size 1073741824" . this work when the endpoint configuration is not serverless. is there any documentation around image/container size for aws managed images?

已提問 2 年前檢視次數 721 次
1 個回答
1

It sounds like you set up a serverless endpoint with 1GB of memory and the image is larger than that. You can increase the memory size of your endpoint with the MemorySizeInMB parameter, more info in this documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints-create.html#serverless-endpoints-create-config

If you pick a larger value for that (e.g. 4096 MB) then it should hopefully work.

AWS
Heiko
已回答 2 年前
  • @Heiko - thanks, I tried with the max as well , i.e. 6 GB. I still get same error message.

  • @Heiko - also , when i create the endpoint configuration as Provisioned instead of serverless , it doesn't complain about the image size.

  • I just realised (it was hard to see without the thousand separators) that the image you're pulling is close to 16GB (I initially thought it was 1.6GB). Because it is 16 GB, even a config with 6GB memory won't be enough. It also makes sense that a provisioned instance doesn't complain as a provisioned instance has much more memory than a serverless endpoint.

    Can I ask the reason why you try to pick the image manually? Just asking because the Sagemaker API can pick the right image for you: https://sagemaker.readthedocs.io/en/stable/api/utility/image_uris.html

  • Example:

    region = boto3.session.Session().region_name
    
    image_uri = sagemaker.image_uris.retrieve(
        framework='huggingface',
        base_framework_version='pytorch1.7',
        region=region,
        version='4.6',
        py_version='py36',
        instance_type='ml.m5.large',
        image_scope='inference'
    )
    

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南