Kubernetes Metrics Server setup recommendations

Kubernetes Metrics Server (KMS) is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines. It is used to implement the Metrics API metrics.k8s.io which is used by Horizontal Pod Autoscaler (HPA).

HPA is implemented as a controller, a control loop, in kube-controller-manager and calculates the desired number of replicas pods of a K8s deployment using an HPA object based on metrics like CPU and memory provided by KMS and the defined thresholds in the HPA object.

So KMS is a critical component in any K8s/Amazon EKS clusters, because without an available and scalable metrics API, HPA controller is not able to properly calculate the number of replicas. This can lead to either reduced workload performance, because scale-up does not work, or increased costs, because scale-in does not work, because of missing metrics.

Open source KMS by default comes as a deployment with just one replica. For High Availability it is recommended to run 2 replicas. The recommended kube-apiserver CLI flag --enable-aggregator-routing=true is an Amazon EKS default setting. Metrics Server collects resource metrics from kubelet's of all worker nodes. So it does not make sense to increase the number of replicas beyond 2 to avoid additional load on kubelet's. On March 31, 2025 Amazon announced the Amazon EKS community Add-ons catalog which contains the metrics-server add-on, which by default comes with 2 replicas.

To prohibit metrics API unavailability it is recommended to use podAntiAffinity to place the pods on different nodes and use a PodDisruptionBudget to control the number of interruptions by either specifying minAvailable or maxUnavailable replicas . The Amazon EKS community add-on metrics-server comes with podAntiAffinity and a PodDisruptionBudget with maxUnavailable=1 by default.

In addition put KMS on nodes, which are not interrupted frequently. At least avoid that both KMS replicas are disrupted at the same time. With Karpenter, a popular open source node scaler, one can make use of NodePool's spec.disruption.budgets to control the number of nodes being interrupted simultaneously. The same applies for EKS Auto Mode which uses Karpenter under the hood.

As the number of nodes and pods grows, the number of metrics and metrics consumers grows. It is recommended to follow the Scaling guidelines from upstream K8s to properly adjust the CPU and memory resources of KMS pods according to the number of nodes and pods in the K8s/Amazon EKS cluster to guarantee good performance of KMS.