EKS pods get node IP address instead of IP from Calico IP Pool
in order to take advantage of higher pod density along with network policies we removed AWS VPC CNI add-on (aws-node) and rely on Calico CNI add-on (calico-node) for pod network management. After removing aws-node daemonset and terminating EKS worker nodes, new nodes were created by ASG and all pods scheduled.
My question, while most pods scheduled on the new nodes have been assigned IP address from Calico IP pool, as expected, a number of pods, mostly daemonset pods, have been assigned the same IP as the node IP.
Any help with this will be highly appreciated. JS
Can you inspect and see if the pod spec has hostNetwork set to true?
Jason_S, thanks for that. hostNetwork spec is set to true in pods with node IP, indeed.
Obviously I'm new to AWS EKS and CNI in general. I guess hostNetwork is set to true on purpose, for example, calico-node pods require direct access to host network ?
This is due to a limitation of EKS (Unable to deploy Calico to control plane nodes), you can refer here https://projectcalico.docs.tigera.io/getting-started/kubernetes/managed-public-cloud/eks. Generally speaking hostNetwork is a bad idea from a security point of view and only trusted pods should have it enabled (even that is not recommended).
Additionally, not sure what pod density you are concerned about. For performance and reliability perspective we strongly discourage you from exceeding the limit such as in here https://github.com/awslabs/amazon-eks-ami/blob/master/files/eni-max-pods.txt. However if it's an ENI imposed limit (i.e. # of ENIs attached to the instance) you can refer to the following blogpost - https://aws.amazon.com/blogs/containers/amazon-vpc-cni-increases-pods-per-node-limits/
Jason_S, many thanks for excellent answer. That helps a lot. Much appreciated.
Verify foreign Elastic IPAccepted Answerasked 2 months ago
Connection refused by public ip address, although image pulled from ECR running appropriately in pod containerasked 3 months ago
Amazon EKS service IP addressesasked 3 months ago
Reserve a pool of Public IP address from AWS for an accountAccepted Answerasked 2 years ago
Get Amplify app IP network adressasked 2 years ago
Private IP address change, effect(s)?asked 2 years ago
Public ip address connectivity in AWSasked 3 months ago
Can't migrate Classic Elastic IP with Reverse DNS to VPC Elastic IPasked 7 months ago
EKS pods get node IP address instead of IP from Calico IP PoolAccepted Answerasked 2 months ago
About dynamic change of IP addressasked 23 days ago