How to remove a specific node from EKS node group (none managed eksctl)

0

I have a small EKS cluster created by eksctl. It consists of 2 node groups (none managed).

The first group runs the management app and currently consists of a single node, the second group runs the "worker" pods and has 3 nodes.

I have received an event notification that the EC2 machine that is the single node in the management nodegroup is scheduled to be shutdown due to maintenance. My plan is to scale the group to 2 nodes and then use kubectl drain to evict the running application so it will move to the new node. All good so far.

My question is how to then remove the original node before EC2 shuts it down (and presumably starts a replacement)? Is there a way to specify a specific node when scaling the group back down to a single node with eksctl? Or will it be smart enough to pick the node that is already drained/cordoned?

asked 2 years ago10289 views
1 Answer
1
Accepted Answer

I understand one of the EKS node is scheduled for the maintenance. Currently you have second group that runs the "worker" pods and has 3 nodes. You want to scale in the group to 2 nodes and want the node scheduled for the maintenance should are removed as part of scale in. You have workload already transferred to other 2 node at moment.

Below is list of steps I have tested in my test environment to achieve this.

You might already be aware you have autoscaling group for the group and it takes care of launching/termination of node and keep the node count as per you autoscaling desired capacity.

  • Drain on EKS:

$ kubectl get nodes

$ kubectl cordon <node name>

$ kubectl drain <node name> --ignore-daemonsets

To terminate the instance and also decrements the size of the Auto Scaling group.

$ aws autoscaling terminate-instance-in-auto-scaling-group --instance-id <INSTANCE_ID> --should-decrement-desired-capacity --region <REGION>

Refer: https://docs.aws.amazon.com/cli/latest/reference/autoscaling/terminate-instance-in-auto-scaling-group.html

AWS
SUPPORT ENGINEER
Sishu_R
answered 2 years ago
profile pictureAWS
EXPERT
Toni_S
reviewed 2 years ago
  • Thanks, If you can give the folk at weave a poke, this would be really nice to wrap up in the eksctl command.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions