Delete Inference Component Sagemaker

Question

Hello !
I'm trying to delete some inference components from deleted endpoints.
I was trying differents methods by aws cli or boto3.

```
aws sagemaker list-inference-components --region eu-central-1
{
    "InferenceComponents": [
        {
            "CreationTime": 1707619639.152,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/huggingface-pytorch-tgi-inference-2024-02-11-02-1707619638-aa40",
            "InferenceComponentName": "huggingface-pytorch-tgi-inference-2024-02-11-02-1707619638-aa40",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v2-summarization-endpoint",
            "EndpointName": "dev-epostbox-v2-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707620610.616
        },
        {
            "CreationTime": 1707539285.172,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-3",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-3",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-endpoint",
            "EndpointName": "dev-epostbox-v1-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707548094.115
        },
        {
            "CreationTime": 1707075798.484,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-1",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-1",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-endpoint",
            "EndpointName": "dev-epostbox-v1-summarization-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707081941.992
        },
        {
            "CreationTime": 1707023267.455,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/DEMO-1707023266-6b96-inference-component-0",
            "InferenceComponentName": "DEMO-1707023266-6b96-inference-component-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/demo-1707022974-d5b4-endpoint",
            "EndpointName": "DEMO-1707022974-d5b4-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707029439.106
        },
        {
            "CreationTime": 1706980504.464,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0",
            "InferenceComponentName": "dev-epostbox-v1-summarization-inference-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/dev-epostbox-v1-summarization-model",
            "EndpointName": "dev-epostbox-v1-summarization-model",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1707031949.226
        },
        {
            "CreationTime": 1706788995.558,
            "InferenceComponentArn": "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/DEMO-1706788994-407e-inference-component-0",
            "InferenceComponentName": "DEMO-1706788994-407e-inference-component-0",
            "EndpointArn": "arn:aws:sagemaker:eu-central-1:904977793552:endpoint/demo-1706784905-e5b4-endpoint",
            "EndpointName": "DEMO-1706784905-e5b4-endpoint",
            "VariantName": "AllTraffic",
            "InferenceComponentStatus": "Failed",
            "LastModifiedTime": 1706800841.762
        }
    ]
}
```
When I'm trying to delete one of the inference components I receive the error

aws sagemaker delete-inference-component --inference-component-name dev-epostbox-v1-summarization-inference-0

An error occurred (ValidationException) when calling the DeleteInferenceComponent operation: Could not find endpoint "dev-epostbox-v1-summarization-model".

What should I do?

Answer

Understood on your latest comment, Alex (no endpoint in service).  Have you tried using the ARN instead?

```
aws sagemaker delete-inference-component --inference-component-name "arn:aws:sagemaker:eu-central-1:904977793552:inference-component/dev-epostbox-v1-summarization-inference-0"
```

I would also ensure that you have the latest copy of the [AWS CLI version 2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

If the above solves the issue, please choose this as the Accepted Answer so others on re:Post may benefit - Thank you!

Answer

Hi,

Your objects seem to be inference endpoints.

So, can you try `aws sagemaker delete-endpoint` ?

See https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/delete-endpoint.html

Best,

Didier

Answer

Hi, Alex

Try the preliminary steps included below also.

1. List and Delete the Endpoint Configurations

```
aws sagemaker list-endpoint-configs
aws sagemaker delete-endpoint-config --endpoint-config-name 
```

2. List the Endpoints

```
aws sagemaker list-endpoints
```

3. List the inference components

```
aws sagemaker list-inference-components
```

4. If you want to filter only the endpoint-associated inference component, you can use the option like below:

```
aws sagemaker list-inference-components --endpoint-name 
```

5. Delete the inference component

```
aws sagemaker delete-inference-component --inference-component-name 
```

It will take a minute or two to completely delete it.

You can check whether it is deleted with the command in step4 (list the inference component).

6. Delete the endpoint

```
aws sagemaker delete-endpoint --endpoint-name 
```

Delete Inference Component Sagemaker

Relevant content