How to host 100 ingress objects on EKS using ALB

Question

We have an application deployed in EKS that dynamically registers ingress rules in ALB.

Each ingress rule maps to a distinct hostname on a common domain (eg `foo-001.example.com` `foo-002.example.com` etc).

At the moment we are hitting the ALB Target Group limit of 100 as each ingress rule is creating both an ALB rule *and* an ALB Target Group. We have had the rule limit increase to 200, but the Target Group limit cannot be changed.

Is there are way to reuse/share Target Groups when creating the EKS ingress objects?

We currently use the following annotation when creating the ingress object:
```
'alb.ingress.kubernetes.io/target-type': 'ip',
```

The documentation implies changing this to `instance` would then allow us to have one Target Group per k8s node the services are deployed to... but we aren't sure.

This is what we're reading: https://catalog.workshops.aws/eks-immersionday/en-US/services-and-ingress/targetgroupbinding

Answer

Hi, effectively if you change the target to instance instead of ip, then every node will register as target in the targetgroup, so if you have <100 nodes in the cluster this can be a good solution.

You can find more details in the EKS best practice guide : https://aws.github.io/aws-eks-best-practices/networking/loadbalancing/loadbalancing/#use-ip-target-type-load-balancers

If you have > 100 nodes, this will not solve the problem, and instead you could also consider using an internal ingress like nginx, and only register the nginx service pods in the ALB/NLB, and then let the ingress do the routing inside the cluster

How to host >100 ingress objects on EKS using ALB

Relevant content