EKS Node Group Strategy
We are planning to use EKS Nodegroup specific applications which will be running on one EKS cluster like below .
Lets consider , if we can consider below setup - application are bound to specific node groups. EKS total Nodes - 4 ,
EKS application namespace - 2 (app1 & app2 ) ,
EKS Node group - 2 ,
EKS node per node group - 2
Question is
- Where it is best practice/recommended patterns to create node group ?
- if we create node groups per applications ( logical isolation - namespace & hard isolation - node group ) which can help each application can scale and running independently without effecting each other - is there any side effect for this ?
It would he very help if someone can provide insight of EKS nodegroup and what purpose we can use.
- Newest
- Most votes
- Most comments
It is important to know the goals that you need to achieve by creating multiple node groups. Possible goals are
- physical app segregation by creating worker node group per application type (the one depicted in your diagram)
- Ensure high availability of worker nodes in case of AZ failure by creating minimal worker node groups
- Cost optimization by allowing multiple application co-exist in the same group of worker nodes
- Node groups to provide different memory to CPU ratio (compute and memory optimized worker node groups). High cpu intensive apps can be deployed on cpu optimized worker node group.
- Node groups based on reservation vs spot instances. Application which tolerate failures can be deployed on spot instances node group
Based on the goal, you need to design the node-group.
Additionally, you can consider EKS Fargate with well defined memory and cpu "requests".
It is difficult to recommend a best practice for how you isolate your applications and provision your node groups, without data about the nominal resource footprints of your applications, and the affect these applications have, under load, on your node, kubelets, and cluster API server. That said, application isolation to nodes is doable and may be recommended when the aggregate collection of applications, running on the same nodes, causes your nodes to thrash, kubelets to load the API server with queries, and evict or constantly autoscale pods and nodes. Node isolation is generally used for multi-tenancy and data classification isolation (PCI, etc.). For data classification, cluster isolation, network segmentation, and even account isolation is also used.
All that said, isolating to node groups is not the only way. You may want to look into Karpenter. I have included some links below:
- https://karpenter.sh/
- https://aws.amazon.com/blogs/containers/scaling-kubernetes-with-karpenter-advanced-scheduling-with-pod-affinity-and-volume-topology-awareness/
- https://aws.amazon.com/blogs/containers/managing-pod-scheduling-constraints-and-groupless-node-upgrades-with-karpenter-in-amazon-eks/
- https://aws.amazon.com/blogs/containers/using-amazon-ec2-spot-instances-with-karpenter/
If you do decide to isolate applications to specific nodes or node groups, then I would also recommend this EKS Best Practices Guide document on multi-tenant isolation with Policy-as-Code for mutation and validation.
Relevant content
- asked 9 months ago
- Accepted Answerasked 2 years ago
- asked a year ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated a year ago
Thanks for your prompt and brief reply. So let me summarize the goals -
Cost Optimization - One NodeGroup for all application type Nodegroup based on fault tolerance type - for fault tolerant workload like batch can be use Spot instance and critical application will go for reservation instance node group
Resource Optimization - More than one NodeGroups based on type of application CPU intensive vs memory vs general load
Resource Isolation & HA - Node group per Application type create separate node groups for each critical application
Please feel to add any other goals we can think for node group decisions.
So, I have two public website NodeJS angular applications which are both critical and deployed on same cluster. Then what would best node group strategy? I would say - each application will have separate node group. Need to understand it has any impact?
it would be helpful if anyone can address above queries ?