EKS Karpenter nodeclaim error

Question

i am using eks version 1.31 with auto-mode. i created a nodepool referencing default nodeclass for spot instance. i also created service-linked-role . But the nodeclaim is showing status as given below and no node is attached to cluster. Why is this and how to fix this?

```
Status:
  Conditions:
    Last Transition Time:  2025-02-15T12:53:03Z
    Message:               object is awaiting reconciliation
    Observed Generation:   1
    Reason:                AwaitingReconciliation
    Status:                Unknown
    Type:                  Initialized
    Last Transition Time:  2025-02-15T12:53:03Z
    Message:               Error creating fleet
    Observed Generation:   1
    Reason:                LaunchFailed
    Status:                Unknown
    Type:                  Launched
    Last Transition Time:  2025-02-15T12:53:03Z
    Message:               Node not registered with cluster
    Observed Generation:   1
    Reason:                NodeNotFound
    Status:                Unknown
    Type:                  Registered
    Last Transition Time:  2025-02-15T12:53:03Z
    Message:               Initialized=Unknown, Launched=Unknown, Registered=Unknown
    Observed Generation:   1
    Reason:                ReconcilingDependents
    Status:                Unknown
    Type:                  Ready
Events:                    
```

And this is my nodepoool

```
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot
  annotations:
    kubernetes.io/description: "NodePool for provisioning spot capacity"
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64", "amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: eks.amazonaws.com/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: eks.amazonaws.com/instance-generation
          operator: Gt
          values: ["4"]
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
```

re:Post Agent · Accepted Answer

The issue you're experiencing with your EKS Karpenter NodeClaim not attaching nodes to your cluster could be due to several factors. Let's analyze the problem and suggest some solutions:

1. NodeClaim Status: The status you've provided indicates that the node creation process is failing at the launch stage. The "LaunchFailed" reason suggests there might be an issue with creating the EC2 instance.

2. EC2 Instance Launch: Check if there are any EC2 instance limits in your AWS account that might be preventing new instances from being launched. Also, ensure that the instance types specified in your NodePool are available in your region and compatible with your VPC settings.

3. IAM Permissions: Verify that the IAM roles associated with your EKS cluster and Karpenter have the necessary permissions to create and manage EC2 instances. The service-linked role you created should have the appropriate permissions, but double-check to ensure nothing is missing.

4. VPC and Networking: Ensure that your VPC is correctly configured and that there are available IP addresses in the subnets where Karpenter is trying to launch instances. Also, check if there are any network ACLs or security groups that might be blocking necessary traffic.

5. Karpenter Logs: To get more insight into the issue, you should check the Karpenter logs. In EKS Auto Mode, you don't have direct access to Karpenter logs, but you can use the Kubernetes NodeDiagnostic resource to retrieve node logs or use the AWS EC2 CLI command 'get-console-output' to get console output from the instances that failed to join the cluster.

6. NodePool Configuration: Your NodePool configuration looks correct, but ensure that the default NodeClass it's referencing is properly configured and exists in your cluster.

7. Cluster Endpoint: Make sure your EKS cluster's API server endpoint is reachable from within the VPC where the nodes are being launched.

8. AWS STS Endpoint: If you're running in a private VPC without internet access, ensure you have set up an AWS STS VPC endpoint, as Karpenter uses IAM Roles for Service Accounts (IRSA) which require access to the STS service.

To troubleshoot further:

1. Use 'kubectl describe nodeclaim <nodeclaim-name>' to get more details about the specific NodeClaim that's failing.
2. Check Kubernetes events with 'kubectl get events --sort-by=.metadata.creationTimestamp' for any relevant error messages.
3. Review the status of your pods to see if they're pending or if there are any other issues that might be preventing node creation.

If after checking these points you're still unable to resolve the issue, you may need to contact AWS support for more detailed diagnostics, as they have access to the underlying Karpenter logs and infrastructure in EKS Auto Mode.  
**Sources**  
[Troubleshoot EKS Auto Mode - Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/auto-troubleshoot.html)  
[How to debug Auto-Mode custom NodePool | AWS re:Post](https://repost.aws/questions/QUWczMGy6MQ4WdctpHA0xkGA/how-to-debug-auto-mode-custom-nodepool)  
[Karpenter - Amazon EKS](https://docs.aws.amazon.com/eks/latest/best-practices/karpenter.html)

EKS Karpenter nodeclaim error

Add your answer

Relevant content