AWS EKS ami sandbox_image url in version 1.30

The issue you're facing with EKS version 1.30 and the sandbox image (pause:3.8) being pulled from registry.k8s.io rather than AWS ECR is indeed related to the way EKS has been configured for sandbox images in this newer version. In earlier versions like EKS 1.28, the pause container image was hosted in Amazon ECR, which is accessible by default for instances with appropriate IAM permissions in an EKS environment.

Options to Resolve the Issue Without a Proxy Server:

1. Override the Sandbox Image to Use ECR:

   • You can configure the kubelet on your worker nodes to pull the sandbox image from Amazon ECR instead of registry.k8s.io. This approach avoids the need for a proxy server.
   • To do this, you would modify the kubelet configuration on your worker nodes to point to the ECR-hosted pause image. This can be done by customizing the bootstrap.sh script in your Launch Template.

   Here’s how you could modify the script:

   bash sed -i 's|registry.k8s.io/pause:3.8|602401143452.dkr.ecr.ap-southeast-1.amazonaws.com/eks/pause:3.5|' /etc/eks/bootstrap.sh

   This command changes the default sandbox image to the ECR-hosted image that’s accessible within your environment.

   Steps:

   1. Create a custom AMI by launching an instance using the EKS AMI, then modify the bootstrap.sh script with the above sed command.
   2. Create an image of this instance and use it in your Launch Template for the self-managed node group.

2. Mirror the pause Image in ECR:

   • Another approach is to manually pull the pause:3.8 image from registry.k8s.io, tag it, and push it to your own ECR repository. Then, configure the kubelet to pull the image from your private ECR.

   Steps:

   1. Pull the pause:3.8 image to your local machine or an environment with internet access: bash docker pull registry.k8s.io/pause:3.8
   2. Tag the image for your ECR repository: bash docker tag registry.k8s.io/pause:3.8 <your_account_id>.dkr.ecr.<region>.amazonaws.com/eks/pause:3.8
   3. Push the image to your ECR repository: aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <your_account_id>.dkr.ecr.<region>.amazonaws.com docker push <your_account_id>.dkr.ecr.<region>.amazonaws.com/eks/pause:3.8
   4. Update the bootstrap.sh script in your nodegroup launch template or custom AMI to use the ECR-hosted image: bash sed -i 's|registry.k8s.io/pause:3.8|<your_account_id>.dkr.ecr.<region>.amazonaws.com/eks/pause:3.8|' /etc/eks/bootstrap.sh

3. Use SSM to Update Nodes After Launch:

   • If you do not want to create a custom AMI, you can use an AWS Systems Manager (SSM) automation or a user data script in your launch template to apply the above sed command to modify the bootstrap.sh script after the node launches.

4. Allow Access to registry.k8s.io via NAT Gateway:

   • If modifying the sandbox image isn’t feasible, consider allowing your nodes access to the internet via a NAT Gateway. This way, they can pull images from registry.k8s.io directly.

   • Steps:

     1. Ensure that your VPC has a NAT Gateway configured.
     2. Attach the route table for the private subnet where your worker nodes reside to the NAT Gateway.
     3. Confirm that the worker nodes can reach registry.k8s.io.

To resolve the issue without setting up a proxy server, you can either override the pause image in the bootstrap.sh script to point to an ECR-hosted image, mirror the pause:3.8 image in your own ECR, or ensure that your nodes have internet access via a NAT Gateway.