Skip to content

Why did changing the IAM role name for my Windows node group fix ALB target health issues in EKS?

0

I have an Amazon EKS cluster running version 1.33 with two node groups (Windows and Linux).

My Windows nodes use a custom AMI based on the EKS-optimized Windows 1.32 AMI (which comes with kube-proxy preinstalled).

Initially, I attached the same IAM role to both node groups.

When I did this, the Windows application pods showed as unhealthy in the ALB target group, even though the pods themselves were running.

To troubleshoot, I created a new IAM role with the exact same permissions (only the role name changed) and attached it to the Windows node group. After doing this, the ALB target group reported the Windows pods as healthy and everything started working fine.

Why would simply changing the IAM role (keeping the same permissions) resolve this issue? Does EKS or aws-auth behave differently when multiple node groups share the same IAM role?

1 Answer
0

The issue you experienced with ALB target health for Windows pods was likely related to how the IAM role was configured in the aws-auth ConfigMap or access entries in your EKS cluster.

When multiple node groups share the same IAM role, there can be complications with how the AWS IAM Authenticator maps these roles to Kubernetes RBAC permissions. The aws-auth ConfigMap is critical for allowing worker nodes to join the cluster by mapping IAM roles to Kubernetes groups.

One common issue is that the AWS IAM Authenticator doesn't permit paths in role ARNs used in the ConfigMap. If your original role configuration included a path, this could have caused authentication problems.

Another possibility is that the node role mapping in the aws-auth ConfigMap wasn't correctly configured. Worker node instance roles must be properly mapped to either the system:bootstrappers or system:nodes groups - not to system:masters (which would be inappropriate for node roles).

When you created a new role with a different name but the same permissions, this likely forced a refresh of the aws-auth ConfigMap or access entries, creating a clean mapping without any previous configuration issues.

Additionally, the kubelet daemon on your Windows nodes needs specific permissions to describe EC2 resources in your VPC and pull container images from Amazon ECR. If there were any subtle issues with how these permissions were being applied or recognized with the shared role, creating a dedicated role for the Windows nodes may have resolved these permission recognition issues.

For best practices, it's generally recommended to use separate IAM roles for different node groups, especially when they run different operating systems like Windows and Linux, to maintain clearer separation of permissions and avoid potential conflicts in role mappings.
Sources
Troubleshooting IAM - Amazon EKS
Troubleshoot Amazon EKS node group in Degraded status | AWS re:Post
Amazon EKS node IAM role - Amazon EKS

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.