Karpenter NodeOverlays in Practice: Three Scenarios You'll Actually Use

TL;DR

• Karpenter makes instance selection and replacement decisions based on price, but the only prices it references are the AWS Pricing API (On-Demand public price) and the real-time Spot price.
• Account-level discounts like Savings Plans or Reserved Instances, per-instance license costs, and custom resources exposed via Device Plugins are not included in this calculation.
• NodeOverlay fills this gap by overriding the price or Extended Resources during the scheduling simulation.
• This post walks through three scenarios with samples:
  1. Reflecting the effective unit cost (SP / RI, etc.)
  2. Making Extended Resources visible to the scheduler
  3. Applying a soft penalty to specific instance families

Heads up: NodeOverlay is still Alpha (karpenter.sh/v1alpha1). The schema may change, so if you adopt it in production, make re-validation on every upgrade part of your routine.

---

Prerequisites

1. Verify the CRD

You need a Karpenter version that ships with NodeOverlay, and the CRD must be present.

kubectl get crd nodeoverlays.karpenter.sh
kubectl api-resources | grep -i nodeoverlay

2. Enable the Feature Gate (required)

Because NodeOverlay is an Alpha feature, it is disabled in the default Karpenter configuration. You need to flip settings.featureGates.nodeOverlay to true in the Helm chart to enable it.

Check current state

# Inspect only the featureGates section of the current values
helm get values karpenter -n kube-system --all | yq '.settings.featureGates'

# You can also check the FEATURE_GATES env var on the Karpenter Pod
kubectl -n kube-system get pod -l app.kubernetes.io/name=karpenter \
  -o jsonpath='{.items[0].spec.containers[0].env}' \
  | jq '.[] | select(.name=="FEATURE_GATES")'

Enable via Helm

Using Helm CLI --set

#
# - `--reuse-values`: keeps the existing values as-is. Without this flag, required settings
#   like `clusterName` and `interruptionQueue` may be reset to chart defaults.
# - `--version`: pins to the currently installed version. If omitted, the chart auto-upgrades
#   to the latest version, which upgrades Karpenter itself along with it.

# Check the currently installed chart version first (to pin the upgrade)
CHART_VERSION=$(helm list -n kube-system -o json \
  | jq -r '.[] | select(.name=="karpenter") | .chart' \
  | sed 's/karpenter-//')

helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter \
  --version "${CHART_VERSION}" \
  --namespace kube-system \
  --reuse-values \
  --set settings.featureGates.nodeOverlay=true

Using a karpenter-values.yaml file

...
settings:
  clusterName: {CLUSTER_NAME}
  featureGates:
    nodeOverlay: true
...

# Pin the current chart version
CHART_VERSION=$(helm list -n kube-system -o json \
  | jq -r '.[] | select(.name=="karpenter") | .chart' \
  | sed 's/karpenter-//')

helm upgrade karpenter oci://public.ecr.aws/karpenter/karpenter \
  --version "${CHART_VERSION}" \
  --namespace kube-system \
  -f karpenter-values.yaml

Verify the change

# Confirm that NodeOverlay=true is present in FEATURE_GATES on the restarted Pod
kubectl -n kube-system get pod -l app.kubernetes.io/name=karpenter \
  -o jsonpath='{.items[0].spec.containers[0].env}' \
  | jq '.[] | select(.name=="FEATURE_GATES")'

Changing a feature gate triggers a restart of the Karpenter controller Pod. New provisioning and consolidation pause briefly during the restart, so it's best to avoid high-traffic windows.

3. Test EC2NodeClass / NodePool

Use the YAML below. Replace CLUSTER_NAME with your own cluster name.

01-ec2nodeclass.yaml

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: overlay-test
spec:
  amiFamily: AL2023
  role: "KarpenterNodeRole-${CLUSTER_NAME}"
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${CLUSTER_NAME}"
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${CLUSTER_NAME}"
  amiSelectorTerms:
    - alias: al2023@latest
  tags:
    Purpose: karpenter-nodeoverlay-test

02-nodepool.yaml

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: overlay-test
spec:
  template:
    metadata:
      labels:
        purpose: nodeoverlay-test
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: overlay-test
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand", "spot"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m5.large", "m5.xlarge", "m5.2xlarge", "m6i.large", "m6i.xlarge"]
      expireAfter: 720h
  limits:
    cpu: "64"
    memory: 256Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s

---

Scenario 1. Reflect the Effective Unit Cost (SP / RI) in Scheduling

Background

Karpenter does not see Savings Plans or Reserved Instances attached to your account. SP and RI are billing-layer discounts that aren't exposed through the EC2 API, so there's no way for them to reach Karpenter's Pricing Provider.

That leads to situations like this:

• You prepay a 1-year Compute SP for the m5 family → effectively ~30% off
• Karpenter looks at the public price and decides "m6i is a bit cheaper," so it provisions m6i
• Result: SP utilization drops, and m6i runs at the full On-Demand price

For reference, ODCR (On-Demand Capacity Reservation) and Capacity Block for ML have been natively recognized by Karpenter since v1. Those are "reserved capacity" and therefore surface through the EC2 API.

Correcting with NodeOverlay

Apply priceAdjustment: "-30%" to the m5 family that has the SP so that Karpenter "believes" m5 is cheap.

03-overlay-sp-discount.yaml

# Reflect the effective discount rate of the Savings Plan covering the m5 family
apiVersion: karpenter.sh/v1alpha1
kind: NodeOverlay
metadata:
  name: overlay-m5-sp-discount
spec:
  weight: 10
  requirements:
    - key: node.kubernetes.io/instance-type
      operator: In
      values: ["m5.large", "m5.xlarge", "m5.2xlarge"]
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["on-demand"]
  priceAdjustment: "-30%"

Limitations

• Pricing vs. Billing are separate. priceAdjustment is only a scheduling hint and has no effect on your actual AWS invoice. The official docs are explicit about this: "Price adjustments influence Karpenter's scheduling decisions but don't affect actual cloud provider billing."
• Coverage exhaustion isn't detected. Usage beyond the SP commit actually runs at the full On-Demand price, but the overlay will keep treating it as cheap.
• Manual management. You have to adjust the overlay value by hand whenever commitments renew or expire, or when you shift generations.

Instead of priceAdjustment, you can also use spec.price to override with an absolute value. The two fields are mutually exclusive, so pick one: priceAdjustment for discount-rate semantics, price when you want to say "treat this instance's unit price as exactly X."

---

Scenario 2. Make Extended Resources Visible to the Scheduler

Background

Things like HugePages, smarter-devices/fuse, or custom Device Plugin resources (e.g. custom-hardware/accelerator) aren't part of the default instance metadata. So when a Pod requests one of these resources, Karpenter concludes:

"No instance type here can host this Pod."

And it either fails to provision, or scheduling ends up tangled in unexpected ways.

Correcting with NodeOverlay

Use the capacity field to tell Karpenter, "pretend this instance type has this Extended Resource." Karpenter then factors that into the simulation when picking candidate instances.

04-overlay-extended-capacity.yaml

# Tell the simulation that the m5 family has smarter-devices/fuse and hugepages-2Mi
apiVersion: karpenter.sh/v1alpha1
kind: NodeOverlay
metadata:
  name: overlay-extended-capacity
spec:
  weight: 10
  requirements:
    - key: node.kubernetes.io/instance-type
      operator: In
      values: ["m5.large", "m5.xlarge", "m5.2xlarge"]
  capacity:
    smarter-devices/fuse: 1
    hugepages-2Mi: 100Mi

Heads up: Simulated ≠ Actual. capacity only affects Karpenter's scheduling simulation. It does not create those resources on the actual node. You still need to configure the real resources through a Device Plugin, a DaemonSet, HugePages in UserData, etc. Skip that step and you get the mismatch where Karpenter picks and launches the right node, but the Pod stays Pending with "insufficient resources."

Also, the capacity field is for Extended Resources only. You cannot override standard resources like CPU, memory, or storage.

---

Scenario 3. Apply a Soft Penalty to Specific Instances

Background

Sometimes you don't want to say "don't use this," but rather "prefer the other option first, and fall back to this one only if needed." For example:

• Protect GPU instances: if CPU workloads scale out and land on GPU nodes, the GPUs go to waste.
• Reflect license costs: when Windows or commercial DB licenses add a fixed per-instance cost, the real price is higher than the public price.
• Guide a generation transition: penalize the older family so workloads gradually move to the newer one.

You could achieve something similar with Taint/Toleration, but that's a binary "allowed/not allowed" control. A price-based penalty is handled as a priority, which is a better fit for expressing a soft preference.

Implementing with NodeOverlay

Set a positive value on priceAdjustment and the target instance looks more expensive, so Karpenter deprioritizes it.

05-overlay-soft-penalty.yaml

# Add a license cost to m6i.xlarge (absolute +0.50)
apiVersion: karpenter.sh/v1alpha1
kind: NodeOverlay
metadata:
  name: overlay-license-fee
spec:
  weight: 5
  requirements:
    - key: node.kubernetes.io/instance-type
      operator: In
      values: ["m6i.xlarge"]
  priceAdjustment: "+0.50"
---
# Apply a +15% penalty to the older m5 family to nudge workloads toward m6i
apiVersion: karpenter.sh/v1alpha1
kind: NodeOverlay
metadata:
  name: overlay-legacy-family-penalty
spec:
  weight: 5
  requirements:
    - key: node.kubernetes.io/instance-type
      operator: In
      values: ["m5.large", "m5.xlarge", "m5.2xlarge"]
  priceAdjustment: "+15%"

---

Stacking Multiple Overlays: Conflict Resolution

When several overlays match the same instance type, the following rules apply:

• Higher weight wins. Default is 0; larger numbers take precedence.
• Equal weights are broken by alphabetical order of the overlay name.
• Field-level behavior differs.
  • price / priceAdjustment: override (the higher-weight value replaces the lower-weight one)
  • capacity: merge (capacity maps from different overlays are combined)
• Unresolvable conflicts (e.g. equal weight with different priceAdjustment values) surface as Ready=False on status.conditions, and the overlay is not applied.

Worked example: SP discount + legacy penalty

overlay-m5-sp-discount from Scenario 1 (-30%) and overlay-legacy-family-penalty from Scenario 3 (+15%) both target the m5 family. Their weights are 10 and 5 respectively, so the SP discount wins and the legacy penalty is ignored. If you want both effects to land, you can't simply leave them at the same weight; combine them into a single priceAdjustment value (e.g. -19.5% as a rough -30% + +15% blend), or accept that one of the two intents is dropped.

---

Operational View: Consolidation and Change Propagation

NodeOverlay changes are automatically incorporated into Karpenter's consolidation logic.

• Price changes alter cost calculations, which can trigger replacement of existing nodes.
• capacity changes are also factored in when deciding whether workloads can be moved between nodes.
• Updates roll out on the next consolidation cycle. No drift-forcing or manual node recreation is required.

Practical caution: if you already have many m5 nodes running and you add a fresh -30% to m5, Karpenter re-evaluates "m5 is now cheaper" and may replace some m6i nodes with m5. To avoid a large replacement wave, give consolidateAfter some headroom, or roll out weight changes gradually.

---

Debugging: Checking Status

You can verify whether an overlay is actually applied by inspecting its status:

kubectl get nodeoverlay
kubectl describe nodeoverlay <name>

Key conditions:

• Ready=True: the overlay is successfully applied to matching instance types.
• Ready=False: configuration conflicts or requirement mismatches prevented application. Check status.conditions for reason / message (e.g. reason: "Conflict", message: "conflict with another overlay").

---

Wrap-up

NodeOverlay lets you feed real-world context (discounts, licenses, custom resources) into Karpenter's price-based decision engine. If you need to customize how Karpenter chooses nodes, it's worth considering.

References

• Karpenter NodeOverlays official docs
• Karpenter NodePools