Demystifying EFS-Claim not bound on AWS Fargate with Amazon EKS

AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. AWS Fargate is compatible with both Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS). With Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers.

What is StatefulSets ?

StatefulSet is a workload API object designed to assist in managing stateful applications. It can manage the scaling and deployment of a collection of pods and provide guarantees about the uniqueness and ordering of these pods. StatefulSet can help you handle storage volumes that provide persistence. If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution.

Error:

When a Fargate pod using PVC is created and the claim hasn't bounded yet, the pod will stay in pending status and will not retry to be scheduled after. After the PV creation, the PVC changed its status to "bound", but pod will still be pending with same error in events as shown below:

Reproduction Steps:

Step-1. Create PV claim and pod at the same time.

Step-2. Post Step-1, Create the PV.

Conclusion:

The reason for the pod stuck in the pending state is because for PVC, System do a validation check for one-time using the validators (on Fargate Scheduler). System then checks, if the PVC is bound for the pod which is created. If it is not bound, pod will be in Rejected state and in this case, status of the pod will be mentioned as "POD_UNSUPPORTED_ON_FARGATE". The re-conciliation loop will take max duration of an hour to schedule the pod on Fargate node which is in pending state.

This reason is mentioned in the EFS CSI driver document(1). User is requested to not to proceed on the pod creation until the STATUS of the PVC is bound.

(1) https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

Co-Authored by: Sidhartha Kotha