How do I resolve the DELETE_FAILED error when deleting the capacity provider in Amazon ECS?

Short description

If you try to delete a capacity provider for your cluster using either the AWS CLI or an API, you might receive one of the following errors:

• "updateStatus": "DELETE_FAILED"
• "updateStatusReason": "The capacity provider cannot be deleted because it is associated with cluster: your-cluster-name. Remove the capacity provider from the cluster and try again."

You might receive these errors for the following reasons:

• The capacity provider that you're trying to delete is in use by an Amazon ECS service in the capacity provider strategy. The AWS Management Console doesn't let you delete a capacity provider that's in use by an Amazon ECS service. In this scenario, you receive this error message: "The specified capacity provider is in use and cannot be removed" in the console. You can disassociate an existing capacity provider from a cluster only if it's not in use by any existing tasks. If you run the DeleteCapacityProvider AWS CLI command, then the capacity provider transitions into DELETE_FAILED status. To resolve this issue, complete the steps in the Check if your capacity provider is in use by an Amazon ECS service in the capacity provider strategy section.
• Your capacity provider is used by the default strategy. If you don't choose a capacity provider strategy or launch type when you run a task or create a service, then a capacity provider strategy is associated with your cluster by default. However, the association happens only if the capacity provider is set as the default capacity provider strategy for the cluster. You can delete only the capacity providers that aren't associated with a cluster. To resolve this issue, complete the steps in the Check if your capacity provider is set in the default capacity provider strategy for the cluster section.

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.

Resolution

Check if your capacity provider is in use by an Amazon ECS service in the capacity provider strategy

1.    If you have multiple services in a cluster, then use the following script to verify the services that are using the capacity provider.

Before you run the script, do the following:

Set the cluster and capacity provider parameters to your values. Set your AWS CLI credentials to your AWS Region. Install jq from the jq website.

#! /bin/bash
cluster=clustername 
capacityprovider=capacityprovidername
services=$(aws ecs list-services --cluster ${cluster} | jq --raw-output '.serviceArns[]')
aws ecs describe-services \
    --cluster ${cluster} \
    --services ${services} \
    | jq -r --arg capacityprovider "${capacityprovider}" \
    '.services[] | select(.capacityProviderStrategy[]?.capacityProvider == $capacityprovider) | .serviceName'

Note: If the script returns a blank output, then none of the services in the cluster are using the capacity provider. Skip to the Check if your capacity provider is set in the default capacity provider strategy for the cluster section.

2.    Update the services that are returned in the output from the script with a new capacity provider.

3.    Delete the old capacity provider.

Important: You can't update a service using a capacity provider strategy or launch type. You must update the service with another capacity provider.

Check if your capacity provider is set in the default capacity provider strategy for the cluster

1.    To find the default capacity provider for your cluster, run the following command:

$ aws ecs describe-clusters --cluster mycluster | jq '.clusters[].defaultCapacityProviderStrategy'
[
  {
    "capacityProvider": "oldCP",
    "weight": 0,
    "base": 0
  }
]

2.    To delete the capacity provider, you must modify the default capacity provider strategy for your cluster using either the Amazon ECS console or the AWS CLI.

Using the Amazon ECS console:

1.    Open the Amazon ECS console.

2.    In the navigation pane, choose Clusters, and then choose your cluster.

3.    Choose Update Cluster.

Using the AWS CLI:

$ aws ecs put-cluster-capacity-providers \
     --cluster mycluster \
     --capacity-providers newCP \
     --default-capacity-provider-strategy capacityProvider=newCP \
     --region us-east-1

$ aws ecs delete-capacity-provider --capacity-provider oldCP

$ aws ecs describe-capacity-providers --capacity-provider oldCP

Note: In the preceding code example, replace mycluster with your cluster. Replace newCP with the new capacity provider that you want to add. Replace oldCP with the capacity provider that you want to delete.

4.    Delete the old capacity provider.

Any existing capacity providers associated with a cluster that are omitted from the PutClusterCapacityProviders API call are disassociated from the cluster. The same rules apply to the cluster's default capacity provider strategy.