How do I troubleshoot namespaces in a terminated state in my Amazon EKS cluster?

2 minute read
1

I tried to delete a namespace in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster. However, the namespace is stuck in the "Terminating" status.

Short description

To delete a namespace, Kubernetes must first delete all the resources in the namespace. Then, it must check registered API services for the status. A namespace gets stuck in Terminating status for the following reasons:

  • The namespace contains resources that Kubernetes can't delete.
  • An API service has a False status.

Resolution

1.    Save a JSON file like in the following example:

kubectl get namespace TERMINATING_NAMESPACE -o json > tempfile.json

Note: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.

2.    Remove the finalizers array block from the spec section of the JSON file:

"spec": {
        "finalizers": [
            "kubernetes"
        ]
    }

After you remove the finalizers array block, the spec section of the JSON file looks like this:

"spec" : {
    }

3.    To apply the changes, run the following command:

kubectl replace --raw "/api/v1/namespaces/TERMINATING_NAMESPACE/finalize" -f ./tempfile.json

Note: Replace TERMINATING_NAMESPACE with the name of your stuck namespace.

4.    Verify that the terminating namespace is removed:

kubectl get namespaces

Repeat these steps for any remaining namespaces that are stuck in the Terminating status.

Related information

What is Amazon EKS?

Namespaces (on the Kubernetes site)

AWS OFFICIAL
AWS OFFICIALUpdated a year ago
2 Comments

Your friendly one-liner. Make sure you don't terminate the wrong thing. I put in a safety check to make sure the variable is at least not empty. Just set TERMINATING_NAMESPACE at the beginning (I had a few to delete, hence the automation).

TERMINATING_NAMESPACE=bad-namespace-here && echo "Force Terminating: ${TERMINATING_NAMESPACE}..." && echo "${TERMINATING_NAMESPACE}" != "" && kubectl get namespace ${TERMINATING_NAMESPACE} -o json | jq 'del(.spec.finalizers)' | kubectl replace --raw "/api/v1/namespaces/${TERMINATING_NAMESPACE}/finalize" -f -
Rob
replied 9 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 9 months ago