EKS Fargate Deployment: Pods Pending with "No Nodes Available" Error

Question

I recently deployed an EKS Fargate cluster using the provided manifest file with eksctl. However, after deployment, I noticed that my pods were stuck in a pending state. Here's the manifest file I used:

```
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: eks-fargate
  version: "1.28"
  region: us-west-2

fargateProfiles:
  - name: my-fargate-profile
    podExecutionRoleArn: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    selectors:
      - namespace: default
      - namespace: kube-system
      - namespace: dev

vpc:
  id: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  cidr: "XXXXXXXXXXXXXXXX"
  subnets:
    public:
      us-west-2a: { id: subnet-XXXXXXXXXXXXXXXX }
      us-west-2b: { id: subnet-XXXXXXXXXXXXXXXX }
      us-west-2c: { id: subnet-XXXXXXXXXXXXXXXX }

private:
      us-west-2a: { id: subnet-XXXXXXXXXXXXXXXX }
      us-west-2b: { id: subnet-XXXXXXXXXXXXXXXX }
      us-west-2c: { id: subnet-XXXXXXXXXXXXXXXX }

```

When I check the status of the pods using kubectl get po -A, I see that the pods in the kube-system namespace, specifically coredns, are stuck in a pending state:

```
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
kube-system   coredns-7864b4f64c-qpzt6   0/1     Pending   0          62m
kube-system   coredns-7864b4f64c-rqqnc   0/1     Pending   0          62m

```

Further investigation using kubectl describe deployment coredns -n kube-system reveals the following events:

```
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  109s (x14 over 67m)  default-scheduler  no nodes available to schedule pods

```

It seems like there are no nodes available to schedule the pods. What could be the cause of this issue, and how can I resolve it? Any insights or suggestions would be greatly appreciated.

Answer

Could you double-check the Fargate profile configuration to ensure that it includes the correct selectors for the **kube-system** namespace. Ensure that the podExecutionRoleArn is correct and has the necessary permissions to launch pods in the specified namespaces.

could you also check similar issue  that was resolved here :- https://repost.aws/questions/QU-eT9qSuzQaiDj8S8yVwoxQ/eks-cluster-with-fargate-type-pod-going-to-pending-state
https://repost.aws/knowledge-center/eks-resolve-pending-fargate-pods

Ensure that the kube-system namespace has the appropriate labels that match the selectors defined in the Fargate profile. Fargate profiles use namespace selectors to determine which namespaces should use Fargate for pod execution.

Check Node Availability: Verify that there are nodes available in your cluster. You can use the command kubectl get nodes to list the nodes in your cluster and their current status.

Inspect Node Conditions: Review the conditions of the nodes in your cluster using kubectl describe nodes. Look for any conditions or issues that might prevent the nodes from scheduling pods, such as resource exhaustion or node taints.

Review Resource Requests and Limits: Ensure that the resource requests and limits specified in the pod manifests are appropriate for the nodes in your cluster. Pods may fail to schedule if their resource requirements exceed the capacity of the available nodes.

Check Node Selector and Affinity: If your pods have node selectors or node affinity rules defined, verify that they match the labels of the available nodes in your cluster. Pods with specific node affinity requirements may fail to schedule if there are no nodes that meet those requirements.

Check Network and Storage: Ensure that the network and storage configurations of your nodes are functioning correctly. Issues with networking or storage can prevent pods from being scheduled.

Review Cluster Autoscaler Configuration: If you're using a cluster autoscaler, review its configuration to ensure that it's configured correctly and has the necessary permissions to scale up the cluster if needed.

Answer

May be similar -https://repost.aws/knowledge-center/eks-resolve-pending-fargate-pods

EKS Fargate Deployment: Pods Pending with "No Nodes Available" Error

Relevant content