Error when deleting and Inference Endpoint in Sagemaker


I deploy a Mistral 7b model using AWS Sagemaker Jump start. I deleted model, endpoint config but when delete endpoint it is an error doesn't allow me to delete inference endpoint. It says " Cannot delete endpoint with inference component associated. Please delete inference component and try it again." I searched but I can not find any solution for this problem, please help me. the instance I deploy is g5.4xlarge and I have to pay $70/day for this problem.

Enter image description here

preguntada hace 3 meses1040 visualizaciones
2 Respuestas
Respuesta aceptada


You are currently experiencing an issue where endpoints created using SageMaker Jump Start are not deleted.

The Inference component that is connected to the endpoint remains, and it occurs this issue.

You can resolve this issue in the following order through AWS CLI:

  1. (if necessary) AWS CLI Update

[+] Install or update the latest version of AWS CLI

  1. List and Delete the Endpoint Configurations
aws sagemaker list-endpoint-configs
aws sagemaker delete-endpoint-config --endpoint-config-name <config_name>
  1. List the Endpoints
aws sagemaker list-endpoints
  1. List the inference components
aws sagemaker list-inference-components

If you want to filter only the endpoint-associated inference component, you can use the option like below:

aws sagemaker list-inference-components --endpoint-name <endpoint_name>
  1. Delete the inference component
aws sagemaker delete-inference-component <component_name>

It will take a minute or two to completely delete it.

You can check whether it is deleted with the command in step4 (list the inference component).

  1. Delete the endpoint
aws sagemaker delete-endpoint --endpoint-name <endpoint_name>

Plus, you can also use boto3 to follow these steps.

If I have missed anything or answered wrong, please feel free to ask me again. Also you have any questions, comment please!

profile picture
respondido hace 3 meses
  • Thank you very much. You saved my day.

  • Point 5 is missing the --inference-component-name flag. Should be aws sagemaker delete-inference-component --inference-component-name <component_name>


Hello ! I'm trying to delete an inference component. But everytime I'm trying to delete the inference component after 2-3 minutes, when I check again I receive failure.

{ "InferenceComponents": [ { "CreationTime": "2024-02-03T19:15:04.464000+02:00", "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0", "InferenceComponentName": "dev-epostbox-v1-summarization-inference-0", "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-model", "EndpointName": "dev-epostbox-v1-summarization-model", "VariantName": "AllTraffic", "InferenceComponentStatus": "Failed", "LastModifiedTime": "2024-02-04T08:46:45.171000+02:00" } ] }

How should I proceed? Thanks, Alex

respondido hace un mes

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas