How do I update the properties in Amazon EKS managed node groups?

Short description

The process to update a managed node group property depends on the property type that you want to change.

You can use UpdateNodegroupConfig or UpdateNodegroupVersion to update the following properties:

• labels
• taints
• scalingConfig
• nodeRepairConfig
• updateConfig
• launchTemplate
  Note: To edit the launchTemplate property, you must specify it when you create the managed node group.

You can use custom launch templates to update the following properties:

• diskSize
• remoteAccess
• tags
  Note: You can also use the TagResource API to update tags.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

To change the instanceType property, specify the instance type in the launch template when you create the managed node group. Then, you can update it in the launch template.

To change the amiType property, specify the ImageId in the launch template when you create the node group. Then, you can update it to use a different Amazon Machine Image (AMI) ID in the launch template.

To update properties that you can't directly modify, you must first create a managed node group. Then, optionally set a Pod Disruption Budget (PDB), drain the existing nodes, and delete the old managed node group.

Create a new managed node group

Complete the following steps:

1. Create a new managed node group.
2. Check whether the nodes that launched from the managed node group that you created successfully joined the cluster. Run the following command:

   kubectl get node -l 'eks.amazonaws.com/nodegroup=my-new-node-group-name'

   Note: Replace my-new-node-group-name with the desired name for your new node group.
   If the nodes joined the cluster successfully, then you see an output similar to the following:

   NAME                            STATUS   ROLES    AGE     VERSION
   ip-192-168-95-10.ec2.internal   Ready    <none>   3m33s   v1.33.3-eks-3abbec1
   ip-192-168-96-53.ec2.internal   Ready    <none>   3m32s   v1.33.3-eks-3abbec1

Optional: Set PDBs

To safely migrate a Pod to a newly created managed node group and maintain its availability, it's a best practice to set PDBs for workloads.

If you already use a PDB, then check the configuration. Run the following command:

kubectl get pdb -n your_namespace

Note: Replace your-namespace with the name of your namespace.

Example output:

NAME       MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
my-pdb     2               N/A               2                     95d

If you haven't configured a PDB, then you can use the following template to create one for each workload.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: mypdb
  namespace: your_namespace
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: myapp

Note: Replace your_namespace with the name of your namespace.

For more information, see Protect critical workload with Pod Disruption Budgets.

Drain existing nodes

Mark the existing node as not schedulable, and then evict the Pods that run on it.

Note: If you use Kubernetes Cluster Autoscaler, then scale the deployment down to zero replicas to avoid conflicts in scaling actions. Run the following command:

kubectl scale deployments/cluster-autoscaler --replicas=0 -n kube-system

Complete the following steps:

1. Prevent new Pods from being scheduled. Run the following command:

   kubectl cordon -l 'eks.amazonaws.com/nodegroup=my-old-node-group-name'

   Note: Replace my-old-node-group-name with your existing node group name.
2. Remove workloads from each unschedulable node one at a time. Run the following command:

   kubectl drain --ignore-daemonsets --delete-emptydir-data your_node

   Note: Replace your-node with the name of your node.
3. Verify that the evicted Pods run on the new nodes. Run the following command:

   kubectl get deploy -n your_namespace

   Note: Replace your_namespace with the name of your namespace.
   Example output:

   NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/test                 4/4     4            4           5d3h
   

Delete the old managed node group

After all Pods except DaemonSet are evicted from the existing node, delete the old managed node group.

Related information

Kubernetes Data Plane