not able to add sagemaker dependencies as external dependencies to lambda

0

Hi Team,
I'm trying to package sagemaker dependencies as external dependcies to upload to lambda.
But I'm getting the max size limit error. Package size is more than allowed size limit i.e.. deployment package size is 50 MB.
And the reason I'm trying to do this is, 'get_image_uri' api is not accessible with boto3.
sample code for this api :
#Import the get_image_url utility function Amazon SageMaker Python SDK and get the location of the XGBoost container.

import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri
container = get_image_uri(boto3.Session().region_name, 'xgboost')

Any reference would be of great help. Thank you.

demandé il y a 5 ans692 vues
3 réponses
0
Réponse acceptée

Could you explain in more detail why do you want to have sagemaker inside of a lambda please?

répondu il y a 5 ans
0

Requirement is to train sagemaker model from lambda. So in the trainingconfiguration we have to pass the container for algorithm image.
so container definition can either have hardcoded image uris and regions mapping, or, as below code snipper where you can get the latest image for specified region and algorithm.

container = get_image_uri(boto3.Session().region_name, 'xgboost')

so to access the above api 'get_image_uri' I need to do below import.
import sagemaker
from sagemaker.amazon.amazon_estimator import get_image_uri

This is not supported by default in lambda.so I'm trying to package sagemaker dependencies as external dependencies and deploy in lambda.

répondu il y a 5 ans
0

As s3 allows more size than direct upload, I even tried with uploading from s3 path.
But since sagemaker has numpy dependency and default numpy doesn't work with python containers, I had to add customized numpy packages for aws lambda, which raised the compressed file size to 105MB.
Now that's also not working as it supports only upto 100MB and uncompressed size of less than 250MB.
and reference or suggestion would be of great help.Thank you.

répondu il y a 5 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions