- Newest
- Most votes
- Most comments
Please follow the below steps to troubleshoot your error:
Step 1. Execute kubectl describe deploy coredns -n kube-system
and check the Pod template section. Does it show something like:
Pod Template:
Labels: eks.amazonaws.com/component=coredns
k8s-app=kube-dns
Annotations: eks.amazonaws.com/compute-type: ec2
Service Account: coredns
if there is annotations field in the deployment please execute the command:
kubectl patch deployment coredns \ -n kube-system \ --type json \ -p='[{"op": "remove", "path": "/spec/template/metadata/annotations/eks.amazonaws.com~1compute-type"}]'
If there if no annotation field like below:
Pod Template:
Labels: eks.amazonaws.com/component=coredns
k8s-app=kube-dns
Service Account: coredns
then proceed to step 2 as executing the patch command again will result in error as we are trying to remove the annotations field from the deployment which does not exist.
Step 2 - Execute - k describe pod <your-coredns-podname> -n kube-system
and look for Events field, i.e. Is it showing something like below?
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 41s (x2 over 6m) default-scheduler no nodes available to schedule pods
Based on the error above, you will be able to find the reason of why coredns pods are not able to schedule.
If suppose the error is something similar to above, please create a fargate profile to target coredns pods like this below (replace my-cluster with your cluster name, 111122223333 with your account ID, AmazonEKSFargatePodExecutionRole with the name of your Pod execution role, and 0000000000000001, 0000000000000002, and 0000000000000003 with the IDs of your private subnets. If you don't have a Pod execution role, you must create one first. ):
aws eks create-fargate-profile \
--fargate-profile-name coredns \
--cluster-name my-cluster \
--pod-execution-role-arn arn:aws:iam::111122223333:role/AmazonEKSFargatePodExecutionRole \
--selectors namespace=kube-system,labels={k8s-app=kube-dns} \
--subnets subnet-0000000000000001 subnet-0000000000000002 subnet-0000000000000003
If you are not able to troubleshoot based on the above, please let me know and I would be happy to help based on your comments.
Thanks, Manish
Hi Manish,
I am also getting same error while I attempt to patch coredns -
$ kubectl patch deployment coredns -n kube-system --type json -p='[{"op": "remove", "path": "/spec/template/metadata/annotations/eks.amazonaws.com~1compute-type"}]'
The request is invalid
here are my annotation values -
$ kubectl describe deploy coredns -n kube-system | grep -i Annotations
Annotations: deployment.kubernetes.io/revision: 2
Annotations: kubectl.kubernetes.io/restartedAt: 2023-10-24T11:58:53Z
& here is what the current event says -
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 4m50s fargate-scheduler Pod provisioning timed out (will retry) for pod: kube-system/coredns-55f54c7b9c-wg6zx
Warning LoggingDisabled 4m48s fargate-scheduler Disabled logging because aws-logging configmap was not found. configmap "aws-logging" not found
Warning FailedScheduling 108s fargate-scheduler Pod provisioning timed out (will retry) for pod: kube-system/coredns-55f54c7b9c-wg6zx
Warning LoggingDisabled 106s fargate-scheduler Disabled logging because aws-logging configmap was not found. configmap "aws-logging" not found
k/r Punit
Relevant content
- asked a month ago
- asked 2 years ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated 5 months ago
Hi Manish, I followed the above solution but getting a weird error. . The Deployment has the following annotation -
And when i am trying to remove the annotation with the following command -
I get this weird error-
Could you please help ?
Hi Poonam,
You do not need to remove the annotation "kubectl.kubernetes.io/restartedAt". If your coredns pods are not coming up, could you please execute the command in step 2 and paste the "Message" you get from "Events" field please?
Thanks, Manish