How to check/determine image/container size for aws managed images ?

@Heiko - thanks, I tried with the max as well , i.e. 6 GB. I still get same error message.

@Heiko - also , when i create the endpoint configuration as Provisioned instead of serverless , it doesn't complain about the image size.

I just realised (it was hard to see without the thousand separators) that the image you're pulling is close to 16GB (I initially thought it was 1.6GB). Because it is 16 GB, even a config with 6GB memory won't be enough. It also makes sense that a provisioned instance doesn't complain as a provisioned instance has much more memory than a serverless endpoint.

Can I ask the reason why you try to pick the image manually? Just asking because the Sagemaker API can pick the right image for you: https://sagemaker.readthedocs.io/en/stable/api/utility/image_uris.html

Example:

region = boto3.session.Session().region_name

image_uri = sagemaker.image_uris.retrieve(
    framework='huggingface',
    base_framework_version='pytorch1.7',
    region=region,
    version='4.6',
    py_version='py36',
    instance_type='ml.m5.large',
    image_scope='inference'
)

And here is an example notebook that might be helpful: https://github.com/marshmellow77/nlp-serverless/blob/main/1_model_train_deploy.ipynb