How do I deploy a HuggingFace model on SageMaker AI?

Resolution

Note: Before you begin, make sure that you have either a SageMaker notebook instance or a SageMaker AI Studio domain. 

To deploy a HuggingFace model on SageMaker AI, use either SageMaker AI Jumpstart or the SageMaker AI SDK for Python.

Use SageMaker AI Jumpstart in SageMaker Studio

Complete the following steps:

1. Launch SageMaker Studio.
2. In the navigation pane, choose JumpStart.
3. Choose HuggingFace as the provider.
4. Select your model, and then choose Deploy.
5. Configure your endpoint settings, and then choose Deploy.

Use the SageMaker AI SDK for Python to deploy a model from the HuggingFace hub

Complete the following steps:

1. Open your machine learning environment.

2. Install and upgrade SageMaker AI:

   !pip install --upgrade sagemaker --quiet

3. Start the SageMaker AI session, and then set the runtime role:

   import sagemaker
   sess = sagemaker.Session()
   role = sagemaker.get_execution_role()

4. Define your model parameters:

   from sagemaker.huggingface.model import HuggingFaceModel
   hub = {
     'example-hf-model-id':'distilbert-base-uncased-distilled-squad', # model_id from hf.co/models
     'example-hf-task':'question-answering'
   }

   Note: Replace example-hf-model-id with your model ID from the HuggingFace models list on the Hugging Face website. Replace example-hf-task with the task that you want to use for predictions. For a list of HF_TASK values, see Pipelines on the Hugging Face website.

5. Create the class, and then deploy the class to SageMaker AI:

   huggingface_model = HuggingFaceModel(
      env=hub,
      role=role,
      transformers_version="4.26",
      pytorch_version="1.13",
      py_version='py39',
   )
   predictor = huggingface_model.deploy(
      initial_instance_count=1,
      instance_type="ml.m5.xlarge"
   )

Related information

Hugging Face on the Amazon SageMaker Python SDK website

Resources for using Hugging Face with Amazon SageMaker AI

Deploy a model from the hub on the Hugging Face website

Deploy transformers for inference on the GitHub website