How do I resolve the error that I receive when I write user data for a Bottlerocket AMI-based managed node group in Amazon EKS?

Resolution

If you include incorrect user data in your launch template, then you might receive the following error message:

"Instances failed to join the kubernetes cluster."

The Bottlerocket AMI merges the user data with the Amazon EKS user data so that the nodes can join the cluster. The configuration that you include in your user data overrides the settings that Amazon EKS configures. For more information about the supported settings, see Settings reference on the Bottlerocket website.

The Amazon Elastic Compute Cloud (Amazon EC2) user data in the launch templates that you use with the Bottlerocket AMI-based managed node groups must be in TOML format. For more information, see TOML on the GitHub website. Amazon EKS doesn't support all valid TOML formats. For more information, see Amazon EC2 user data.

The EC2 user data must also include the following Kubernetes settings:

• settings.kubernetes.cluster-certificate
• settings.kubernetes.api-server
• settings.kubernetes.cluster-name

Add the following example user data configuration with the preceding settings in the user data section of your launch template:

[settings.kubernetes]
api-server = "https://abc.def.us-east-1.eks.amazonaws.com"
cluster-certificate = "TkQgQ0VSVElGSUNBVEUtLS0tLQo="
cluster-name = "eks-dev"

Note: Replace us-east-1 with the AWS Region where you deploy your Amazon EKS cluster.

Example of user data with node-labels and node-taints settings:

[settings.kubernetes]
api-server = "https://abc.def.us-east-1.eks.amazonaws.com"
cluster-certificate = "TkQgQ0VSVElGSUNBVEUtLS0tLQo="
cluster-name = "eks-dev"

[settings.kubernetes.node-labels]
"label1" = "foo"
"label2" = "bar"

[settings.kubernetes.node-taints]
"dedicated" = "airflow:NoSchedule"

Note: Replace cluster-name with your cluster name and us-east-1 with your Region.

You can also run the following eksctl command to generate a basic userdata.toml file for your cluster:

~ eksctl get cluster --region us-east-1 --name cluster-name -o json | jq --raw-output '.[] | "[settings.kubernetes]\napi-server = \"" + .Endpoint + "\"\ncluster-certificate =\"" + .CertificateAuthority.Data + "\"\ncluster-name = " + .Name'

Note: Replace cluster-name with your cluster name and us-east-1 with your Region. The command's output includes a prewritten userdata example.

Related information

Bottlerocket file directory on the GitHub website