ALB rules disappear when alternating kubectl and terraform

Question

Months ago I created a Load Balancer Controller ingress with two listeners, 80 and 443, using a Kubernetes manifest. Port 80 has a number of rules attached to it, and port 443 has only one rule attached. The rules forward to target groups based on pods in EKS. This has worked fine.

Just before summer I added some rules to the 443-listener using a terraform script. These rules forward to a target group defined for a network load balancer. During summer and until now this has also worked fine.

So, the problem: if I deploy a new application in EKS using the same load balancer with a new port 80 rule, the terraform-created 443-rules disappears. This also happens if I delete the ingress for one of the 80-rules using kubectl delete. After such new-application or delete operation I have to re-apply the terraform script to get the 443-rules back in. The main question is; what needs to be done in order to make the terraform-generated 443-rules persist? Any help shedding a light on whats going on, or even better, helping me solve the problem will be highly appreciated.

Best regards
globus68-2

Answer

The rules need to have a single threaded owner. What you are seeing is two different Infrastructure as Code mechanism overwriting each others changes. I would recommend managing the listener rules and the NLB itself completely in Terraform and then using the [Load Balancer Controllers TargetGroupBinding](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/targetgroupbinding/targetgroupbinding/) to manage the target groups. This way your listener rules would be controlled from a single source of truth but still integrates with EKS.

ALB rules disappear when alternating kubectl and terraform

Relevant content