Error when deleting and Inference Endpoint in Sagemaker

Question

I deploy a Mistral 7b model using AWS Sagemaker Jump start. I deleted model, endpoint config but when delete endpoint it is an error doesn't allow me to delete inference endpoint.
It says " Cannot delete endpoint with inference component associated. Please delete inference component and try it again."
I searched but I can not find any solution for this problem, please help me.
the instance I deploy is g5.4xlarge and I have to pay $70/day for this problem.

![Enter image description here](/media/postImages/original/IMJ9QIDlRhQaK-2MCjQETkKw)

Accepted Answer

Hello

You are currently experiencing an issue where endpoints created using SageMaker Jump Start are not deleted.

The Inference component that is connected to the endpoint remains, and it occurs this issue.

You can resolve this issue in the following order through AWS CLI:

---

1. (if necessary) AWS CLI Update

[+] [Install or update the latest version of AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

2. List and Delete the Endpoint Configurations

```
aws sagemaker list-endpoint-configs
aws sagemaker delete-endpoint-config --endpoint-config-name 
```

3. List the Endpoints

```
aws sagemaker list-endpoints
```

4. List the inference components

```
aws sagemaker list-inference-components
```

If you want to filter only the endpoint-associated inference component, you can use the option like below:
```
aws sagemaker list-inference-components --endpoint-name 
```

5. Delete the inference component

```
aws sagemaker delete-inference-component 
```
It will take a minute or two to completely delete it.

You can check whether it is deleted with the command in step4 (list the inference component).

6. Delete the endpoint

```
aws sagemaker delete-endpoint --endpoint-name 
```

Plus, you can also use `boto3` to follow these steps.

---
If I have missed anything or answered wrong, please feel free to ask me again. Also you have any questions, comment please!

Answer

Hello ! I'm trying to delete an inference component. 
But everytime I'm trying to delete the inference component after 2-3 minutes, when I check again I receive failure.

{
    "InferenceComponents": [
        {
            "CreationTime": "2024-02-03T19:15:04.464000+02:00",
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-model",
            "EndpointName": "dev-epostbox-v1-summarization-model",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": "2024-02-04T08:46:45.171000+02:00"
        }
    ]
}

How should I proceed?
Thanks,
Alex

Error when deleting and Inference Endpoint in Sagemaker

関連するコンテンツ