EKS node capacity settings with Karpenter

1

Currently we use EKS with managed node groups. In the node group configuration you have to configure the min, max and desired capacity of the nodes.

With the use of the classic Cluster Autoscaler the desired node capacity of the autoscaling group will be updated. Now we switched to the AWS developed autoscaler Karpenter (https://karpenter.sh/). Karpenter has a different handling and does not work with the autoscaling group or node group.

So my question is, with the use of Karpenter, which is the best setting for the EKS min, max and desired capacity? Should we set all these to "0" for example?

질문됨 2년 전3649회 조회
1개 답변
1

Karpenter doesn't actually scale out nodes within the Managed Node Group. While you can use Karpenter alongside managed node groups, Karpenter defines its own set of limits (provisioner.spec.limits.resources). Kube scheduler will schedule to existing nodes in a managed node group if they exist. But when scaling, it will spin up new nodes that are not part of any Autoscaling Group or Managed Node Group. Hopefully that helps!

AWS
Jeff_W
답변함 2년 전
  • +1

    Karpenter by design is different than cluster autoscaler. It aims at rapid scale-out for unscheduled pods and also quick scale-in (ttl based) when nodes are empty. The focus is now back on making compute available on-demand and fast.

  • Ideally, Karpenter should work in harmony with existing node groups (managed or not) by manipulating ASGs directly rather than creating unmanaged instances. Perhaps an alternate Provisioner could be developed to do this. It would be less complex than the existing Provisioner since there would be no need for subnetSelector, securityGroupSelector, InstanceProfile, etc. - all can be determined directly from the existing ASG and LaunchTemplate.

  • @kennyk It's actually not quite so easy to work with existing node groups because Karpenter would essentially need to duplicate the logic that Cluster Autoscaler already has to determine which group to scale up and all the additional logic around monitoring that group (which does essentially what you described). There are notable performance improvements in a group-less model, Justin does a better job than I of explaining it https://www.youtube.com/watch?v=3QsVRHVdOnM

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠