Auto Mode CSI errors from detacher

The errors you're seeing are related to the CSI (Container Storage Interface) attacher trying to detach a Persistent Volume (PV) that no longer exists from an instance that also no longer exists. This can happen when resources are deleted but the CSI controller's state hasn't been updated properly.

To resolve these issues in an EKS Auto Mode cluster, you can try the following steps:

1. Verify that the PV and the associated EBS volume have been properly deleted. Use the AWS CLI or console to check if the EBS volume still exists.

2. If the volume still exists, ensure it has the correct tags, especially the 'eks:eks-cluster-name' tag with your cluster name as the value.

3. Check for any lingering VolumeAttachment resources:

   kubectl get volumeattachments

   If you find any that reference non-existent volumes or nodes, you can delete them:

   kubectl delete volumeattachment <name>

4. Restart the EKS Auto Mode components. While you can't directly restart the CSI driver in Auto Mode, you can restart the kube-system pods:

   kubectl delete pods -n kube-system -l k8s-app=kube-proxy

   This will cause the pods to be recreated, potentially refreshing the CSI controller's state.

5. If the issue persists, you may need to recreate the affected nodes. In Auto Mode, this can be done by deleting the NodeClaim associated with the problematic node:

   kubectl get nodeclaims kubectl delete nodeclaim <name>

6. As a last resort, consider upgrading your EKS cluster to the latest version, as newer versions may include fixes for CSI-related issues.

Remember to monitor your cluster after performing these actions to ensure the errors don't reoccur. If the problem continues, you may need to contact AWS support for further assistance, as there could be an underlying issue with the EKS Auto Mode implementation.
Sources
EKS Auto Mode starts with 0 nodes & can't schedule new nodes | AWS re:Post
EKS AutoMode node can't bind to static PV - does not contain driver ebs.csi.aws.com | AWS re:Post