not able to add sagemaker dependencies as external dependencies to lambda

0

Hi Team,
I'm trying to package sagemaker dependencies as external dependcies to upload to lambda.
But I'm getting the max size limit error. Package size is more than allowed size limit i.e.. deployment package size is 50 MB.
And the reason I'm trying to do this is, 'get_image_uri' api is not accessible with boto3.
sample code for this api :
#Import the get_image_url utility function Amazon SageMaker Python SDK and get the location of the XGBoost container.

import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri
container = get_image_uri(boto3.Session().region_name, 'xgboost')

Any reference would be of great help. Thank you.

已提問 5 年前檢視次數 692 次
3 個答案
0
已接受的答案

Could you explain in more detail why do you want to have sagemaker inside of a lambda please?

已回答 5 年前
0

Requirement is to train sagemaker model from lambda. So in the trainingconfiguration we have to pass the container for algorithm image.
so container definition can either have hardcoded image uris and regions mapping, or, as below code snipper where you can get the latest image for specified region and algorithm.

container = get_image_uri(boto3.Session().region_name, 'xgboost')

so to access the above api 'get_image_uri' I need to do below import.
import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri

This is not supported by default in lambda.so I'm trying to package sagemaker dependencies as external dependencies and deploy in lambda.

已回答 5 年前
0

As s3 allows more size than direct upload, I even tried with uploading from s3 path.
But since sagemaker has numpy dependency and default numpy doesn't work with python containers, I had to add customized numpy packages for aws lambda, which raised the compressed file size to 105MB.
Now that's also not working as it supports only upto 100MB and uncompressed size of less than 250MB.
and reference or suggestion would be of great help.Thank you.

已回答 5 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南