CoreDNS with ETCD backend on EKS


CoreDNS has a ETCD plugin, which essentially allows for dynamic DNS by reading the values from ETCD.

Since EKS is managed, it means we can't access the etcd instance on master node, that's fine as I can create my own etcd cluster (and I did). Below is my coredns deployment

apiVersion: v1
  Corefile: |
    .:53 {
        kubernetes cluster.local {
          pods insecure
        etcd {
            path /skydns
            endpoint http://etcd-cluster-ip.default.svc.cluster.local:2379
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
kind: ConfigMap
  annotations: {}
  labels: coredns
    k8s-app: kube-dns
  name: coredns
  namespace: kube-system

The issue I face now is that the master node is not able to resolve the ClusterIP DNS etcd-cluster-ip.default.svc.cluster.local, which is the ClusterIP of my etcd cluster. If I change that DNS with the actual ClusterIP, name resolution works as expected and CoreDNS is able to access ETCD

How can the master node resolve the DNS of my cluster ? I see below line in coredns logs

{"level":"warn","ts":"2022-03-16T20:44:42.352Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-fd406ba0-cc21-4132-bfef-ca14e3fd4eb3/etcd-cluster-ip.default.svc.cluster.local:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: lookup etcd-cluster-ip.default.svc.cluster.local on no such host\""}
asked 2 years ago1071 views
1 Answer

I believe you are encountering the same issue as described in this CoreDNS issue -

As per this comment, your desired setup will work only if the FQDN of the ETCD server can be resolved outside the cluster.

The work-around is to directly specify the ClusterIP of your ETCD service in the Corefile endpoint configuration, and I see that you are already using this approach.

The bottom line is that your CoreDNS pod cannot use itself to perform DNS resolution.

I hope this helps!

profile pictureAWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions