Skip to content

Had ECS task with EBS volume stuck in deprovisioning. Was able to "kill" the ECS task, but EBS volume & ENI won't shut down

0

I have volume Id: vol-052db481725ad28e3 and ENI (eni-01c1baa5dc154e20c) in us-east-1. It was originally attached to an ECS task. When attempting to spin down the ECS task, it was stuck in deprovisioning (believe it was originally stuck due to the cluster deleting before the ECS task). I managed to "unstick" the task deprovisioning by creating an ECS cluster with the same name.

Now the Volume and ENI are stuck & unable to be deleted. The Volume and ENI are both active and associated to the original ECS task, but the ECS task is gone / deleted. It's been at least 12 hrs, and I've been unable to find a way to delete these resources.

$ aws ec2 describe-volumes --volume-ids vol-052db481725ad28e3
{
    "Volumes": [
        {
            "Attachments": [
                {
                    "AttachTime": "2024-06-29T14:09:33+00:00",
                    "State": "attached",
                    "VolumeId": "vol-052db481725ad28e3",
                    "DeleteOnTermination": false,
                    "AssociatedResource": "arn:aws:ecs:us-east-1:211125504797:task/ent-dev-cluster/ff75465816a142c1a61bdcd90c618fe1",
                    "InstanceOwningService": "fargate.amazonaws.com"
                }
            ],
            "AvailabilityZone": "us-east-1a",
            "CreateTime": "2024-06-29T14:09:26.102000+00:00",
            "Encrypted": false,
            "Size": 1,
            "SnapshotId": "",
            "State": "in-use",
            "VolumeId": "vol-052db481725ad28e3",
            "Iops": 3000,
            "Tags": [
                {
                    "Key": "AmazonECSCreated",
                    "Value": "arn:aws:ecs:us-east-1:211125504797:task/ent-dev-cluster/ff75465816a142c1a61bdcd90c618fe1"
                },
                {
                    "Key": "AmazonECSManaged",
                    "Value": "true"
                }
            ],
            "VolumeType": "gp3",
            "MultiAttachEnabled": false,
            "Throughput": 125
        }
    ]
}
$ aws ecs describe-tasks --tasks ff75465816a142c1a61bdcd90c618fe1

An error occurred (ClusterNotFoundException) when calling the DescribeTasks operation: Cluster not found.
1 Answer
0

To resolve the issue of deleting the stuck Volume and ENI in AWS, you can try the following steps:

Detach the Volume: Use the AWS CLI to detach the volume from the non-existent ECS task

aws ec2 detach-volume --volume-id vol-052db481725ad28e3

Delete the ENI: Since the ENI is likely still attached to the ECS task, you can try to detach it first and then delete it.

aws ec2 detach-network-interface --attachment-id eni-attach-id
aws ec2 delete-network-interface --network-interface-id eni-01c1baa5dc154e20c

If the attachment ID is not known, you can describe the network interface to get it:

aws ec2 describe-network-interfaces --network-interface-ids eni-01c1baa5dc154e20c

Force Delete the Volume: If the volume is still marked as in-use, you might need to force delete it after detaching it.

aws ec2 delete-volume --volume-id vol-052db481725ad28e3

Check and Clean Up Resources: Ensure there are no lingering dependencies or attachments to the Volume or ENI.

Verify Resource Deletion: After performing the above steps, confirm that the Volume and ENI are no longer present.

aws ec2 describe-volumes --volume-ids vol-052db481725ad28e3
aws ec2 describe-network-interfaces --network-interface-ids eni-01c1baa5dc154e20c

If the resources are still stuck, consider reaching out to AWS Support for assistance in cleaning up the stuck resources.

EXPERT
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.