EKS VPC-CNI Plugin Node Group Setup Questions

Question

I am creating an EKS managed node group in terraform using the eks module version 17.1.0 and up until now specifying the bootstrap_extra_args like so has been working
```
    node_groups = [{
            
            name = "${var.environment}-nodes"
            desired_capacity = var.eks_cluster.desired_capacity
            max_capacity     = var.eks_cluster.max_capacity
            min_capacity     = var.eks_cluster.min_capacity
        
            additional_security_group_ids = aws_security_group.nodes.id
            instance_types                = [var.eks_cluster.node_instance_type]
            key_name = "$$$$$$"
            bootstrap_extra_args = "/etc/eks/bootstrap.sh '${local.cluster_name}' --use-max-pods false --kubelet-extra-args '--max-pods=110'"
    }]
```
I have created two clusters like this and the nodes have been created with the max pods set to 110. both of these clusters are in us-east-2

I am now trying to create a cluster in China region cn-northwest-1 and the same configuration only sets the max pods to 35 and I cannot seem to get it to go any higher.

Node types: t3a.large instances

Note: I have also attempted to launch the nodes in China with a launch_template containing the following userdata script and the script is read, there are no errors that I can find and I end up with the same result.

```
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="//"

--//
Content-Type: text/x-shellscript; charset="us-ascii"
#!/bin/bash -xe
/etc/eks/bootstrap.sh '${cluster_name}' --use-max-pods false --kubelet-extra-args '--max-pods=110'
--//--
```
This begs the question, are eks managed node groups setup a bit differently in china? Is what I'm trying to do even possible without some crazy workaround I cannot seem to find?

Answer

Hello,

Depending on which instance family and size you run, you have a constraint of number of IPs per ENI and a max number of ENI applies too.

As described here : https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI

For example, a t3a.large instance is constrained to 3 ENI and 12 IPs per ENI at max.

Do you use the same instance family/size for your clusters in us-east-2 ?

For ref, a great blog post that explains the mechanisms of CNI IP allocation and how to manage the limits with prefixes.

https://aws.amazon.com/blogs/containers/amazon-vpc-cni-increases-pods-per-node-limits/

Thanks

EKS VPC-CNI Plugin Node Group Setup Questions

Relevanter Inhalt