Skip to content

Does EKS Auto Mode automatically revert modifications to built-in general-purpose NodePool?

-1

Body:

We are running EKS Auto Mode (v1.33) and experienced an issue where modifications to the built-in general-purpose NodePool were automatically reverted by EKS.

What we did:

  • Used kubectl patch to change instance-category from [c, m, r] to [m, r] on the built-in general-purpose NodePool
  • The change was applied successfully at first

What happened:

  • EKS Auto Mode reverted the instance-category back to [c, m, r] within minutes
  • This caused Karpenter to drain m5a.large nodes and provision c5a.large nodes repeatedly
  • The NodePool's .metadata.generation reached 23, indicating this revert cycle happened many times
  • This node churn caused a 3-hour production outage (pods kept restarting on c5a.large nodes)

Questions:

  1. Is this automatic revert behavior intentional/documented? The docs say "You cannot modify the built-in node pools" but doesn't explicitly mention reconciliation/revert behavior.
  2. Does the revert interval differ between EKS versions? (We observed fast reverts on 1.33 but no revert on 1.34 dev cluster during ~10 minutes of testing)
  3. Is using a custom NodePool (with built-in disabled) the recommended approach to control instance-category selection?

Environment:

  • EKS Auto Mode, v1.33 (production), v1.34 (dev)
  • Built-in general-purpose NodePool with managed-by: eks label
  • Region: ap-northeast-2
asked 2 months ago56 views
1 Answer
0

Yes, EKS Auto Mode does automatically revert modifications to built-in NodePools, and this behavior is intentional by design.

Understanding the Revert Behavior:

The built-in NodePools (general-purpose and system) created by EKS Auto Mode are managed resources with the managed-by: eks label. Documentation states that "You cannot modify the built-in node pools," this means AWS EKS actively reconciles these resources back to their intended configuration. When you patch these NodePools, EKS's control plane detects the drift from the expected state and automatically reverts the changes to maintain the default configuration. something what you have experienced.

The rapid revert cycle you experienced is expected reconciliation behavior - each time you or any automation attempted to modify the NodePool, EKS reverts it, incrementing the generation counter.

Recommended Approach:

Yes, using a custom NodePool with the built-in NodePools disabled is the correct and recommended approach when you need to control configuration like instance-category selection. When creating your cluster with Auto Mode, you can disable the default NodePool creation by setting nodePools: [] in your EKS Cluster configuration via Console, CLI or API, then create your own custom NodePools with your desired specifications.

This approach gives you control over:

  • Instance types and categories
  • Disruption budgets and maintenance windows
  • Node lifecycle policies
  • Other NodePool-specific configurations

Regarding Version Differences:

Reconciliation interval differences between EKS versions 1.33 and 1.34 is not documented, but the fundamental principle remains: built-in NodePools are managed resources that should not be modified.

To avoid production issues like the one you experienced, always use custom NodePools for workload-specific requirements rather than attempting to modify the built-in ones.

Sources
EKS Auto Mode - Eksctl User Guide

answered 2 months ago
EXPERT
reviewed 2 months ago
AWS
SUPPORT ENGINEER
revised 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.