EKS VPC-CNI Plugin Node Group Setup Questions

0

I am creating an EKS managed node group in terraform using the eks module version 17.1.0 and up until now specifying the bootstrap_extra_args like so has been working

    node_groups = [{
            
            name = "${var.environment}-nodes"
            desired_capacity = var.eks_cluster.desired_capacity
            max_capacity     = var.eks_cluster.max_capacity
            min_capacity     = var.eks_cluster.min_capacity
        
            additional_security_group_ids = aws_security_group.nodes.id
            instance_types                = [var.eks_cluster.node_instance_type]
            key_name = "$$$$$$"
            bootstrap_extra_args = "/etc/eks/bootstrap.sh '${local.cluster_name}' --use-max-pods false --kubelet-extra-args '--max-pods=110'"
    }]

I have created two clusters like this and the nodes have been created with the max pods set to 110. both of these clusters are in us-east-2

I am now trying to create a cluster in China region cn-northwest-1 and the same configuration only sets the max pods to 35 and I cannot seem to get it to go any higher.

Node types: t3a.large instances

Note: I have also attempted to launch the nodes in China with a launch_template containing the following userdata script and the script is read, there are no errors that I can find and I end up with the same result.

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="//"

--//
Content-Type: text/x-shellscript; charset="us-ascii"
#!/bin/bash -xe
/etc/eks/bootstrap.sh '${cluster_name}' --use-max-pods false --kubelet-extra-args '--max-pods=110'
--//--

This begs the question, are eks managed node groups setup a bit differently in china? Is what I'm trying to do even possible without some crazy workaround I cannot seem to find?

gefragt vor 2 Jahren718 Aufrufe
1 Antwort
0

Hello,

Depending on which instance family and size you run, you have a constraint of number of IPs per ENI and a max number of ENI applies too.

As described here : https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI

For example, a t3a.large instance is constrained to 3 ENI and 12 IPs per ENI at max.

Do you use the same instance family/size for your clusters in us-east-2 ?

For ref, a great blog post that explains the mechanisms of CNI IP allocation and how to manage the limits with prefixes.

https://aws.amazon.com/blogs/containers/amazon-vpc-cni-increases-pods-per-node-limits/

Thanks

AWS
Julien
beantwortet vor 2 Jahren
  • yup I used the same family/size (t3a.large) in us-east-2. Hence my confusion that its behaving differently, but we have also noticed that the console in us-east-2 mentions prefixes for each network interface whereas the console in cn-northwest-1 does not. This makes me think that the prefix assignment feature is not available in the cn-northwest-1 region which could be the issue here possibly?

  • Hi, i don't see any region specific limitation for prefix assignment on my end.

    If you already use this capability without any issue, then i would take a look at VPC CNI logs just to make sure the plugin can operate as expected for allocation.

    ie: troubleshooting scenarios described here : https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen