EKS IPVS mode - error public loadbalancers

Question

Hi 
i have EKS cluster 1.33 with VCNI plugin and kube-proxy plugin. 
IPVS its setup by best pracitcs AWS ( scheduler rr) and i have problem. 
I have several Ingress controllers, all of which have load balancer SVCs. The only difference is in the SVC entry, where the SVC for the public load balancer does not contain an annotation for the internal load balancer and subnet ARN. Otherwise, everything is the same: externalTrafficPolicy local, etc.
If I restart the POD that serves the internal Ingress controller, everything works as expected, with a maximum outage of about 30 seconds before IPVS recalculates. But if I restart the POD that serves public traffic, no traffic goes to it. I also tried nodePorts and got the same result. For internal, curl, wget, etc. work without any problems... I get data, but from the public controller, I get nothing but a connection timeout for this nodePort.
If I switch back to iptables and restart kube-proxy, the public ingress controller starts working, and I can switch back to ipvs, and it continues to work until the next restart. 
The nodes, pods, and load balancers are in the same subnet. 
The controller settings are identical for both public and internal, and I don't understand why nodePort doesn't work when I call it from the same subnet where it is running.

Does AWS already distinguish between public and internal traffic at the nodePort level?

re:Post Agent · Answer

Based on the information available, AWS does not distinguish between public and internal traffic at the nodePort level. The issue you're experiencing with IPVS mode and public load balancers appears to be related to the IPVS configuration itself rather than AWS making distinctions between traffic types.

When you configure kube-proxy to run in IPVS mode, it's a disruptive change that affects how traffic is routed to services. The behavior you're describing - where switching back to iptables mode resolves the issue temporarily - suggests there may be an issue with how IPVS is handling the service endpoints or connection tracking for your public-facing services.

A few things to verify in your setup:

1. Ensure that the kube-proxy DaemonSet was properly restarted after making the IPVS configuration changes. If your worker nodes were joined to the cluster before enabling IPVS, you need to restart the kube-proxy DaemonSet.

2. Verify that the required kernel modules for IPVS are loaded on all worker nodes and that the ipvsadm package is installed.

3. Check that your service endpoints are correctly configured by verifying the endpoint connections in the kube-system namespace.

The fact that your internal load balancer works correctly while the public one doesn't, despite having similar configurations (both using externalTrafficPolicy: Local), suggests the issue might be related to how IPVS is managing the connection state or load balancing for services without the internal load balancer annotations. However, this shouldn't be related to AWS making distinctions at the nodePort level - nodePort functionality is handled by Kubernetes and kube-proxy, not by AWS infrastructure.

You may want to examine the IPVS entries on your worker nodes to see if there are differences in how the public versus internal services are being configured in the IPVS tables.  
**Sources**  
[Running kube-proxy in IPVS Mode - Amazon EKS](https://docs.aws.amazon.com/eks/latest/best-practices/ipvs.html)  
[Troubleshoot AWS Load Balancer Controller errors in Amazon EKS | AWS re:Post](https://repost.aws/knowledge-center/eks-fix-aws-load-balancer-controller-errors)

EKS IPVS mode - error public loadbalancers

Add your answer

Relevant content