How do I automate the HTTP proxy configuration for Amazon EKS with Amazon Linux 2023 worker nodes?

Resolution

In AL2023, NodeConfig uses nodeadm to configure the proxy settings.

Important: To configure HTTP proxy settings through the NodeConfig specification in AL2023, you must have Amazon Machine Image (AMI) release version 20251120 or later.

Create a custom launch template

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Complete the following steps:

1. Create a custom launch template with the AL2023 AMI ID.
   Note: If you don't specify an AMI in your custom launch template, then the managed nodes group automatically merges user data.
2. Configure proxy settings within the user data, and structure your user data with the following fields:

   MIME-Version: 1.0
   Content-Type: multipart/mixed; boundary="//"
   
   --//
   Content-Type: application/node.eks.aws
   
   ---
   apiVersion: node.eks.aws/v1alpha1
   kind: NodeConfig
   spec:
     cluster:
       apiServerEndpoint: API_SERVER_ENDPOINT
       certificateAuthority: YOUR_CLUSTER_CA
       cidr: KUBERNETES_SERVICE_CIDR_RANGE
       name: EKS_CLUSTER_NAME
     instance:
       environment:
         default:
           HTTP_PROXY: http://PROXY_SERVER_IP_ADDRESS:PORT
           HTTPS_PROXY: http://PROXY_SERVER_IP_ADDRESS:PORT
           NO_PROXY: localhost,127.0.0.1,169.254.169.254,VPC_CIDR_RANGE,.internal,.eks.amazonaws.com
   --//--

Note: Replace the following placeholders with your values:

• API_SERVER_ENDPOINT with your API server endpoint
• YOUR_CLUSTER_CA with your cluster certificate authority (CA)
• KUBERNETES_SERVICE_CIDR_RANGE with your service CIDR range
• EKS_CLUSTER_NAME with your cluster name
• PROXY_SERVER_IP_ADDRESS:PORT with your proxy server address and port
• VPC_CIDR_RANGE with your virtual private cloud (VPC) CIDR range

If you use VPC endpoints, then add AWS service endpoints to NO_PROXY. If your cluster endpoint access is only public, then exclude .eks.amazonaws.com from NO_PROXY. For more information about environment variables in NodeConfig, see EnvironmentOptions.

Configure the proxy for aws-node and kube-proxy

Complete the following steps:

1. Configure the proxy based on your cluster endpoint access. If your cluster endpoint access is only public, then configure the proxy for both aws-node and kube-proxy. If your cluster endpoint access is private or public and private without an Amazon Elastic Compute Cloud (Amazon EC2) VPC endpoint, then configure the proxy for aws-node.
2. To create a ConfigMap to configure the environment variables, use the following example script:

   apiVersion: v1
   kind: ConfigMap
   
   metadata:
      name: proxy-environment-variables
      namespace: kube-system
   
   data:
      HTTP_PROXY: http://PROXY_SERVER_IP_ADDRESS:PORT
      HTTPS_PROXY: http://PROXY_SERVER_IP_ADDRESS:PORT
      NO_PROXY: KUBERNETES_SERVICE_CIDR_RANGE,localhost,127.0.0.1,169.254.169.254,VPC_CIDR_RANGE,.internal,.eks.amazonaws.com

   Note: Replace PROXY_SERVER_IP_ADDRESS:PORT with your proxy server address and port. Replace KUBERNETES_SERVICE_CIDR_RANGE with your service CIDR range and VPC_CIDR_RANGE with your VPC CIDR range. If you use VPC endpoints, then add AWS service endpoints to NO_PROXY. If your cluster endpoint access is only public, then exclude .eks.amazonaws.com from NO_PROXY.
3. Apply the ConfigMap to your cluster.
4. To set your HTTP proxy configuration to aws-node and kube-proxy, run the following commands:

   kubectl patch -n kube-system -p '{ "spec": {"template":{ "spec": { "containers": [ { "name": "aws-node", "envFrom": [ { "configMapRef": {"name": "proxy-environment-variables"} } ] } ] } } } }' daemonset aws-node
   kubectl patch -n kube-system -p '{ "spec": {"template":{ "spec": { "containers": [ { "name": "kube-proxy", "envFrom": [ { "configMapRef": {"name": "proxy-environment-variables"} } ] } ] } } } }' daemonset kube-proxy

Create a managed node group

Create a new managed node group for your cluster that uses the custom launch template that you created.

Test your proxy

To verify that your proxy works, complete the following steps:

1. To check the status of your nodes, run the following commands:

   kubectl get nodes
   kubectl run test-pod --image=amazonlinux:2023 --restart=Never -- sleep 300
   kubectl get pods -A
   

   Example output:

   kubectl get nodes
   NAME                           STATUS   ROLES    AGE    VERSION
   ip-10-0-139-233.ec2.internal   Ready       7m7s   v1.32.9-eks-ecaa3a6
   kubectl run test-pod --image=amazonlinux:2023 --restart=Never -- sleep 300
   pod/test-pod created
   kubectl get pods -A
   NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
   default       test-pod                   1/1     Running   0          24s
   kube-system   aws-node-h999l             2/2     Running   0          7m40s
   kube-system   coredns-69bf5c796c-2q478   1/1     Running   0          18h
   kube-system   coredns-69bf5c796c-7lwvt   1/1     Running   0          18h
   kube-system   kube-proxy-sxgn8           1/1     Running   0          7m40s

2. Check your proxy log for additional information on node connectivity.
   Example logs:

   10.0.139.233 TCP_TUNNEL/200 10425 CONNECT auth.docker.io:443 - HIER_DIRECT/XX.XX.XX.XX -
   10.0.139.233 TCP_TUNNEL/200 5285 CONNECT registry-1.docker.io:443 - HIER_DIRECT/XX.XX.XX.XX -
   10.0.139.233 TCP_TUNNEL/200 10424 CONNECT auth.docker.io:443 - HIER_DIRECT/XX.XX.XX.XX -
   10.0.139.233 TCP_TUNNEL/200 54112823 CONNECT docker-images-prod.s3.dualstack.us-east-1.amazonaws.com:443 - HIER_DIRECT/XX.XX.XX.XX -
   10.0.139.233 TCP_TUNNEL/200 5675 CONNECT registry-1.docker.io:443 - HIER_DIRECT/XX.XX.XX.XX -

Related information

Upgrade from Amazon Linux 2 to Amazon Linux 2023

Node bootstrapping