By using AWS re:Post, you agree to the Terms of Use

not able to add sagemaker dependencies as external dependencies to lambda

0

Hi Team,
I'm trying to package sagemaker dependencies as external dependcies to upload to lambda.
But I'm getting the max size limit error. Package size is more than allowed size limit i.e.. deployment package size is 50 MB.
And the reason I'm trying to do this is, 'get_image_uri' api is not accessible with boto3.
sample code for this api :
#Import the get_image_url utility function Amazon SageMaker Python SDK and get the location of the XGBoost container.

import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri
container = get_image_uri(boto3.Session().region_name, 'xgboost')

Any reference would be of great help. Thank you.

asked 3 years ago88 views
3 Answers
0
Accepted Answer

Could you explain in more detail why do you want to have sagemaker inside of a lambda please?

answered 3 years ago
0

Requirement is to train sagemaker model from lambda. So in the trainingconfiguration we have to pass the container for algorithm image.
so container definition can either have hardcoded image uris and regions mapping, or, as below code snipper where you can get the latest image for specified region and algorithm.

container = get_image_uri(boto3.Session().region_name, 'xgboost')

so to access the above api 'get_image_uri' I need to do below import.
import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri

This is not supported by default in lambda.so I'm trying to package sagemaker dependencies as external dependencies and deploy in lambda.

answered 3 years ago
0

As s3 allows more size than direct upload, I even tried with uploading from s3 path.
But since sagemaker has numpy dependency and default numpy doesn't work with python containers, I had to add customized numpy packages for aws lambda, which raised the compressed file size to 105MB.
Now that's also not working as it supports only upto 100MB and uncompressed size of less than 250MB.
and reference or suggestion would be of great help.Thank you.

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions