EKS static IPs for managed node group nodes
To start, I am new to EKS. My customer is asking how to control the egress IPs for nodes in a managed node group in EKS. They have used 'non-managed nodes' (which I think EKS docs call self-managed) but they want to start using a managed node group but they need to control the nodes' IPs to allow setting up firewall rules to restrict on-premises access to the specific pod(s) that are apart of the particular service. They asked if autoscaling of the managed node group makes this a problem and if it does, should they use a managed group without scaling if possible or use a mix of managed and unmanaged nodes. But IPs have to survive a reboot of the underlying node.
My thinking is that the autoscaling is not a problem per se. If they were willing to dedicate specific subnets for this managed node group, we can predict the IPs that way. Let's assume that is not what they want. Next, I was thinking Calico might be able to help control egress IP ranges. But from what I read on a Calico doc, it seems you may need to run Calico in an overlay mode for it to be able to control the egress IPs. It is blurry to me if that is possible for a managed node group but I would guess that won't work. If we could limit traffic to a NAT Gateway, maybe that would solve this but given that you can't use an SG on a NGW, that seems unhelpful. The Calico doc link: https://docs.projectcalico.org/getting-started/kubernetes/managed-public-cloud/eks
In short, can we control the egress IPs of a managed node group easily? And if yes, how exactly? As much info about the how as you can share would be greatly appreciated. To be clear, they consider the cluster to have less trusted services so we cannot just trust everything from the cluster, their firewall has to be able to distinguish traffic from this one service/managed node group.
UPDATE: I've heard an answer for this from a colleague to use PrivateLink with an NLB and then use the NLB IPs, one per subnet, in the firewall rules. See the Hybrid Services section on page 14 of the whitepaper link. https://d1.awsstatic.com/whitepapers/aws-privatelink.pdf
- 최신
- 최다 투표
- 가장 많은 댓글
The easiest way to control the egress IP of a set of pods is to use VPC CNI Custom Networking along with a NAT Gateway.
In the VPC, designate one or more separate internal subnets (ideally one per AZ) in which only these pods live. (There's just no practical way to do this without reserving a subnet.) Create the appropriate ENIConfig CRD to map these pods to the subnets. Then, in one or more separate public subnets in the same AZ, create a NAT Gateway, using a pre-created Elastic IP as its IP address. Create a default route from the each internal subnet to the NAT Gateway in the same AZ.
Is this is still valid comment or there is new feature which can allow to control the egress IP for EKS cluster. I have use case where the protocol is not supported on NAT gateway hence that is not the solution for me.