not able to add sagemaker dependencies as external dependencies to lambda

Question

Hi Team,  
I'm trying to package sagemaker dependencies as external dependcies to upload to lambda.  
But I'm getting the max size limit error. Package size is more than allowed size limit i.e..  deployment package size is 50 MB.  
And the reason I'm trying to do this is, 'get_image_uri' api is not accessible with boto3.  
sample code for this api :   
#Import the get_image_url utility function Amazon SageMaker Python SDK and get the location of the XGBoost container.  
  
import sagemaker  
from sagemaker.amazon.amazon_estimator import get_image_uri  
container = get_image_uri(boto3.Session().region_name, 'xgboost')  
  
Any reference would be of great help. Thank you.

Accepted Answer

Could you explain in more detail why do you want to have sagemaker inside of a lambda please?

Answer

As s3 allows more size than direct upload, I even tried with uploading from s3 path.   
But since sagemaker has numpy dependency and default numpy doesn't work with python containers, I had to add customized numpy packages for aws lambda, which raised the compressed file size to 105MB.   
Now that's also not working as it supports only upto 100MB and uncompressed size of less than 250MB.  
and reference or suggestion would be of great help.Thank you.

Answer

Requirement is to train sagemaker model from lambda. So in the trainingconfiguration we have to pass the container for algorithm image.   
so container definition can either have hardcoded image uris and regions mapping, or, as below code snipper where you can get the latest image for specified region and algorithm.  
  
container = get_image_uri(boto3.Session().region_name, 'xgboost')  
  
so to access the above api 'get_image_uri' I need to do below import.  
import sagemaker  
from sagemaker.amazon.amazon_estimator import get_image_uri  
  
This is not supported by default in lambda.so I'm trying to package sagemaker dependencies as external dependencies and deploy in lambda.

not able to add sagemaker dependencies as external dependencies to lambda

Contenuto pertinente