Patch CoreDNS command is giving error

Question

My EKS Fargate cluster throws error for CoreDNS. I tried the command that is mentioned here: 
https://repost.aws/questions/QUT6VOlwOXSdmvGLIcaGeerw/core-dns-pods-in-kube-system-not-working

I get this error on running the command:

`$ kubectl patch deployment coredns -n kube-system --type=json -p='[{"op": "remove", "path": "/spec/template/metadata/annotations", "value": "eks.amazonaws.com/compute-type"}]'
The request is invalid: the server rejected our request due to an error in our request`

Manish · Accepted Answer

Please follow the below steps to troubleshoot your error:

**Step 1.** Execute `kubectl describe deploy coredns -n kube-system` and check the Pod template section. Does it show something like:

```
Pod Template:
  Labels:           eks.amazonaws.com/component=coredns
                    k8s-app=kube-dns
  Annotations:      eks.amazonaws.com/compute-type: ec2
   Service Account:  coredns
```
if there is annotations field in the deployment please execute the command:

`kubectl patch deployment coredns \
    -n kube-system \
    --type json \
    -p='[{"op": "remove", "path": "/spec/template/metadata/annotations/eks.amazonaws.com~1compute-type"}]'`

If there if no annotation field like below:

```
Pod Template:
  Labels:           eks.amazonaws.com/component=coredns
                    k8s-app=kube-dns
  Service Account:  coredns
```

then proceed to step 2 as executing the patch command again will result in error as we are trying to remove the annotations field from the deployment which does not exist.

**Step 2** - Execute - `k describe pod  -n kube-system` and look for Events field, i.e. Is it showing something like below?

```
Events:
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  41s (x2 over 6m)  default-scheduler  no nodes available to schedule pods
```
Based on the error above, you will be able to find the reason of why coredns pods are not able to schedule.

If suppose the error is something similar to above, please create a fargate profile to target coredns pods like this below (replace my-cluster with your cluster name, 111122223333 with your account ID, AmazonEKSFargatePodExecutionRole with the name of your Pod execution role, and 0000000000000001, 0000000000000002, and 0000000000000003 with the IDs of your private subnets. If you don't have a Pod execution role, you must [create one](https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html#fargate-sg-pod-execution-role) first. ):

```
aws eks create-fargate-profile \
    --fargate-profile-name coredns \
    --cluster-name my-cluster \
    --pod-execution-role-arn arn:aws:iam::111122223333:role/AmazonEKSFargatePodExecutionRole \
    --selectors namespace=kube-system,labels={k8s-app=kube-dns} \
    --subnets subnet-0000000000000001 subnet-0000000000000002 subnet-0000000000000003
```

If you are not able to troubleshoot based on the above, please let me know and I would be happy to help based on your comments.

Thanks,
Manish

cbarillet · Answer

Hello everyone.

It seems that the error changed with Kubernetes 1.30 and fargate today (26 september 2024).
The error was about the taint of the nodes not compatible with the toleration of the coredns pods definition.
I used the following command to be able to start my coredns pods:

> kubectl patch deployment coredns -n kube-system --type json -p='[{"op": "remove", "path": "/spec/template/spec/tolerations"}]'

I hope that it will be helpful for other persons.

Punit Porwal · Answer

Hi Manish,

I am also getting same error while I attempt to patch coredns -

```
$  kubectl patch deployment coredns -n kube-system --type json -p='[{"op": "remove", "path": "/spec/template/metadata/annotations/eks.amazonaws.com~1compute-type"}]'
The request is invalid

```
here are my annotation values -
```
$  kubectl describe deploy coredns -n kube-system | grep -i Annotations

Annotations:            deployment.kubernetes.io/revision: 2
Annotations:      kubectl.kubernetes.io/restartedAt: 2023-10-24T11:58:53Z

```
& here is what the current event says -
```
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  4m50s  fargate-scheduler  Pod provisioning timed out (will retry) for pod: kube-system/coredns-55f54c7b9c-wg6zx
  Warning  LoggingDisabled   4m48s  fargate-scheduler  Disabled logging because aws-logging configmap was not found. configmap "aws-logging" not found
  Warning  FailedScheduling  108s   fargate-scheduler  Pod provisioning timed out (will retry) for pod: kube-system/coredns-55f54c7b9c-wg6zx
  Warning  LoggingDisabled   106s   fargate-scheduler  Disabled logging because aws-logging configmap was not found. configmap "aws-logging" not found
```

k/r
Punit

Patch CoreDNS command is giving error

Relevant content