SageMaker endpoint stuck in "Updating" status

Question

I have a SageMaker inference endpoint that I want to delete, but I'm unable to delete it because it's been stuck in the "Updating" status for weeks at this point.

When I've tried deleting it, I get an error message that says I can't delete an endpoint that's updating: ```Cannot update in-progress endpoint```

I've tried the following:
* Updating the endpoint with a new endpoint configuration to get it out of the Updating status. Didn't work--was unable to complete the update.
* Deleting the endpoint's original endpoint configuration. Didn't work--endpoint is still stuck and unable to be deleted.

Does anyone have any ideas of how this happened and what I can do to force delete the endpoint?

Answer

Hi,
Here are a few suggestions that may help:

1.- **Retry Deleting the Endpoint:** Even though the endpoint is in the "Updating" status, you can still try to delete it. The DeleteEndpoint API operation should be able to delete the endpoint, even if it's in an unusual state. Try deleting the endpoint again and see if it's successful this time.

2.- **Delete the Endpoint Configuration:** If you're still unable to delete the endpoint, try deleting the associated endpoint configuration first. The DeleteEndpointConfig API operation allows you to delete the endpoint configuration, which may then allow you to delete the stuck endpoint. https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-delete-resources.html

3.- **Use the AWS CLI or SDK:** You can try deleting the endpoint and configuration using the AWS CLI or an AWS SDK, such as the AWS SDK for Python (Boto3). This may provide more detailed error messages that could help identify the root cause.

Delete the endpoint using the AWS CLI

*aws sagemaker delete-endpoint --endpoint-name *

Delete the endpoint configuration using the AWS CLI

*aws sagemaker describe-endpoint-config --endpoint-config-name *

4.- **Check for Dependent Resources:** Ensure that there are no other resources, such as models or other configurations, that are still dependent on the endpoint you're trying to delete. These dependencies may be preventing the deletion from completing successfully. Example command to delete model via AWS CLI aws sagemaker describe-endpoint-config --endpoint-config-name 
https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-delete-resources.html

I will recommend to try again, even point 1 and 2, if any of this steps works please contact AWS Support. They can investigate the issue further.

SageMaker endpoint stuck in "Updating" status

Relevant content