How do I use SageMaker AI local mode to test models?

Resolution

To test your model before you deploy it to a production endpoint, you can locally deploy the model on a SageMaker AI notebook instance.

Before you begin, note your model artifacts. Model artifacts are your training job outputs that you store locally or in Amazon Simple Storage Service (Amazon S3).

To deploy your model in local mode, complete the following steps:

1. Install the Amazon SageMaker Python SDK:

   pip install sagemaker

2. In your SageMaker AI notebook instance, run the following commands to import the necessary libraries:

   import boto3
   import sagemaker
   from sagemaker.local import LocalSession

3. To create a model object, run the following command:

   model = Model(
       model_data='s3://your-bucket/path/to/model.tar.gz',
       image_uri='your-container-image-uri',
       role='your-sagemaker-role-arn',
       sagemaker_session=sagemaker_session
   )

   Note: Replace s3://your-bucket/path/path/to/model.tar.gz with your model artifacts and your-container-image-uri with the Amazon Elastic Container Registry (Amazon ECR) image for your framework. Also, replace your-sagemaker-role-arn with your AWS Identity and Access Management (IAM) role and sagemaker_session with your current session.

4. To locally deploy the model, run the following command:

   predictor = model.deploy(initial_instance_count=1,instance_type='local')

   Note: If you're deploying a custom model, then include an entry point script. The script defines how your model loads and handles inference requests.

5. After deployment, run the following command to make inferences:

   result = predictor.predict(your_input_data)
   print(result)

Note: For Amazon SageMaker Studio, you must activate the EnableDockerAccess parameter in the domain's DockerSettings. Then, install Docker for the operating system (OS) of the application. For shell scripts, see amazon-sagemaker-local-mode on the GitHub website. For more information, see Getting started with local mode.

Related information

Local mode on the Amazon SageMaker Python SDK website