Skip to content

eks-pod-identity-agent addon reinstall doesn't recreate the pod-identity-webhook MutatingWebhookConfiguration

0

I deleted the MutatingWebhookConfiguration named pod-identity-webhook from my EKS cluster because Pod Identity injection had stopped working and I was troubleshooting. After deletion, I tried restoring it by reinstalling the eks-pod-identity-agent addon and running:

aws eks update-addon \
  --addon-name eks-pod-identity-agent \
  --resolve-conflicts OVERWRITE

But this didn't recreate it.

The agent itself is fine, so when I manually inject the env vars and projected volume into a pod spec, the agent serves credentials correctly. So the only broken piece is the automatic injection.

env:
  - name: AWS_CONTAINER_CREDENTIALS_FULL_URI
    value: "http://[fd00:ec2::23]/v1/credentials"
  - name: AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE
    value: "/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token"
volumes:
  - name: eks-pod-identity-token
    projected:
      defaultMode: 420
      sources:
        - serviceAccountToken:
            audience: pods.eks.amazonaws.com
            expirationSeconds: 86400
            path: eks-pod-identity-token
volumeMounts:
  - name: eks-pod-identity-token
    mountPath: "/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/"
    readOnly: true

Looking at webhook manifest I had backed up earlier, the clientConfig points to url: https://127.0.0.1:23443/mutate, which seems to be something running on the EKS control plane side rather than the worker nodes. There's no Kubernetes Service backing it, so reapplying the manifest doesn't help. Is this a control plane managed resource? And if so, is there any way to trigger a reconciliation without recreating the cluster?

asked 2 months ago56 views
1 Answer
0
Accepted Answer

The pod-identity-webhook MutatingWebhookConfiguration is a control plane–managed resource within Amazon EKS. If it is manually deleted, it will not be restored by reinstalling the eks-pod-identity-agent addon, as the webhook is provisioned and reconciled by the EKS control plane rather than by the agent pods deployed on worker nodes. The only supported recovery mechanism is to allow the control plane to reconcile the resource, which in practice may require either waiting for reconciliation or recreating the cluster.

https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html

https://github.com/aws/eks-pod-identity-agent

https://registry.terraform.io/providers/-/aws/6.12.0/docs/resources/eks_addon

EXPERT
answered 2 months ago
EXPERT
reviewed 2 months ago
  • Any rough estimate on the reconciliation time? Is it like hours or days?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.