Amazon Elastic Kubernetes Service (EKS) maximum pod calculation with secondary VPC CIDR

Question

Amazon EKS includes scripts in the bootstrap.sh file to pass in the maximum number of pods per instance type based on the number of Interfaces and IPs per interface, and this works well for single CIDR VPCs.

However, it seems that when using a secondary CIDR range, the instance ends up with 1 less ENI available per node, and this can impact pod scheduling.

Does anyone have a solution to this?

References:
 - https://docs.aws.amazon.com/eks/latest/userguide/cni-custom-network.html
 - https://github.com/awslabs/amazon-eks-ami/blob/master/files/bootstrap.sh
 - https://github.com/awslabs/amazon-eks-ami/issues/375

Accepted Answer

Here is a solution that can be implemented placed into the worker node userdata to dynamically calculate the number of pods.

```
# get instance details to feed into bootstrap script`
az=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
instance_type=$(curl -s http://169.254.169.254/latest/meta-data/instance-type)
    
# set region for aws cli
aws configure set region `curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r .region`
    
# calculate number of max-pods
# https://docs.aws.amazon.com/eks/latest/userguide/cni-custom-network.html
max_if=$(aws ec2 describe-instance-types  --instance-types $${instance_type} --query  InstanceTypes[*].NetworkInfo.MaximumNetworkInterfaces --output text)
max_ips_per_if=$(aws ec2 describe-instance-types  --instance-types $${instance_type} --query  InstanceTypes[*].NetworkInfo.Ipv4AddressesPerInterface --output text)
max_pods=$((($${max_if} -1) * ($${max_ips_per_if} -1 ) + 2))
    
/etc/eks/bootstrap.sh  --use-max-pods false --kubelet-extra-args "--node-labels=k8s.amazonaws.com/eniConfig=$${az} --max-pods=$${max_pods}" '${cluster_name}
```

Amazon Elastic Kubernetes Service (EKS) maximum pod calculation with secondary VPC CIDR

관련 콘텐츠