How do I create custom Amazon Linux AMIs for Amazon EKS?

Short description

To create a custom Amazon Linux AMI for Amazon EKS, you must use the following:

• HashiCorp Packer (available from the HashiCorp website)
• A build specification with resources and configuration scripts from the Amazon EKS AMI repository on AWS GitHub

Note: Packer works using an AWS CloudFormation stack. The stack runs an m4.large or a1.large Amazon Elastic Compute Cloud (Amazon EC2) instance (depending on the target AMI architecture). The instance is provisioned by Packer. After the instance is provisioned with packages and binaries, Packer creates an AMI from the running instance.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Install and configure Packer

1.    Install Packer from the HashiCorp website.

2.    Configure your AWS account credentials to allow Packer to make calls to AWS API operations on your behalf. You can use static credentials (secret key and secret access key), an environment variable, shared credential files, or an Amazon EC2 role.

Note: For more information about configuring AWS credentials for Packer, see Authentication and IAM Task or Instance Role on the HashiCorp website.

Clone the Amazon EKS AMI repository

To clone the Amazon EKS AMI repository to your workstation, run the following command:

$ git clone https://github.com/awslabs/amazon-eks-ami && cd amazon-eks-ami

Note: Packer is performed through a series of makefile targets with eks-worker-al2.json as the build specification. The build process uses the amazon-ebs Packer builder (from the HashiCorp website) and launches an instance. The Packer shell provisioner (from the HashiCorp website) runs the install-worker.sh script on the instance to install software and perform other configuration tasks. Then, Packer creates an AMI from the instance and terminates the instance after the AMI is created.

Provide a custom source AMI

To configure a custom source AMI, set the variables source_ami_id, source_ami_owners, and aws_region in the Packer configuration file eks-worker-al2.json. For example:

"source_ami_id": "SOURCE_AMI_ID",      # Enter the ID of your source image
"source_ami_owners": "AWS_ACCOUNT_ID", # Enter the account where this image is stored
"aws_region": "AWS_DEFAULT_REGION",    # Enter the AWS Region of the source AMI

To provide custom worker binaries, complete the steps in the (Optional) Provide your own Kubernetes binaries section.

To build the image using default Kubernetes binaries from AWS, complete the steps in the Build an Amazon EKS worker AMI using default binaries section.

(Optional) Provide your own Kubernetes binaries

When Packer provisions the instance, binaries are downloaded by default from the Amazon EKS public Amazon Simple Storage Service (Amazon S3) bucket amazon-eks in us-west-2. For more information, see the install-worker.sh file.

1.    To examine the available binaries provided in the default bucket, run the following AWS CLI command

$ aws s3 ls s3://amazon-eks 
$ aws s3 ls s3://amazon-eks/kubernetes_version/kubernetes_build_date/bin/linux/arch/

Note: Replace amazon-eks, kubernetes_version, kubernetes_build_date, and arch with your values.

Important: To download your own binaries to the worker node during provisioning, you must mirror the amazon-eks bucket folder structure used in the install-worker.sh script.

2.    After your binaries are prepared through your own build process, copy the binaries to your own S3 bucket using the AWS CLI.

Here's an example that uses a custom kubelet binary:

$ aws s3 cp kubelet s3://my-custom-bucket/kubernetes_version/kubernetes_build_date/bin/linux/arch/kubelet

Note: Replace my-custom-bucket, amazon-eks, kubernetes_version, kubernetes_build_date, and arch with your values.

Important: You must provide all the binaries listed in the default amazon-eks bucket for a specific kubernetes_version, kubernetes_build_date, and arch combination. These binaries must be accessible through AWS Identity and Access Management (IAM) credentials configured in the Install and configure HashiCorp Packer section.

Build an Amazon EKS worker AMI using custom binaries

To start the build process, use the source AMI configured in eks-worker-al2.json to invoke make with parameters. For example:

$ make k8s \
    binary_bucket_name=my-custom-bucket \
    binary_bucket_region=eu-west-1 \
    kubernetes_version=1.14.9 \
    kubernetes_build_date=2020-01-22

Note: Confirm that the binary_bucket_name, binary_bucket_region, kubernetes_version, and kubernetes_build_date parameters match the path to your binaries in Amazon S3.

Build an Amazon EKS worker AMI using default binaries

1.    To build the Amazon EKS worker AMI using a custom base AMI and the default (latest) Kubernetes binaries, confirm that the eks-worker-al2.json file is updated with the correct base AMI.

2.    To trigger the build process by providing the Kubernetes version as the parameter, run the following command:

$ make 1.14  # Build a Amazon EKS Worker AMI for k8s 1.14

Note: For more advanced configurations, you must modify the configuration files in the amazon-eks-ami AWS GitHub repository before triggering the build.

Tip: Check AWS GitHub for common issues with using the Packer configuration to build custom AMIs.