How to reuse SageMaker endpoint created by JumpStartModel.deploy()

0

I have deployed a SageMaker JumpStartModel using this code from a sample notebook:

from sagemaker.jumpstart.model import JumpStartModel
model_id = "meta-textgeneration-llama-2-7b"
pretrained_model = JumpStartModel(model_id=model_id)
pretrained_predictor = pretrained_model.deploy()

However my Jupyter kernel has died and no longer knows the pretrained_predictor variable. How can I re-create the pretrained_predictor from an existing endpoint? I know it's name, for example meta-textgeneration-llama-2-7b-2023-09-10-12-34-56-789 and its ARN if needed.

asked 5 months ago874 views
2 Answers
0

Hi,

To re-use an existing SageMaker endpoit, you have to use the UpdateEndpoint API: see https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateEndpoint.html

Deploys the new EndpointConfig specified in the request, switches to using newly created 
endpoint, and then deletes resources provisioned for the endpoint using the previous 
EndpointConfig (there is no availability loss).

When SageMaker receives the request, it sets the endpoint status to Updating. After 
updating the endpoint, it sets the status to InService. To check the status of an endpoint, 
use the DescribeEndpoint API.

Note
You must not delete an EndpointConfig in use by an endpoint that is live or while the 
UpdateEndpoint or CreateEndpoint operations are being performed on the endpoint. 
To update an endpoint, you must create a new EndpointConfig.

If you delete the EndpointConfig of an endpoint that is active or being created or 
updated you may lose visibility into the instance type the endpoint is using. The endpoint 
must be deleted in order to stop incurring charges.

For Python, the API is detailled here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/update_endpoint.html

Best,

DIdier

profile pictureAWS
EXPERT
answered 5 months ago
  • Thanks, but are you sure about that? I don't want to update the endpoint (ie change it's config), all I want is to tell the Jupyter notebook (or my ML app for that matter) that there is an endpoint created by JumpStartModel.deploy() before and now go and use it. It doesn't seem right that I would have to update_endpoint() every time the app that wants to use it starts..?

0

you can use the predictor class to do this.

AWS
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions