DNS resolution issue with AWS Cloud Map and systemd-resolved

Hi there,

I'm currently facing an issue with DNS resolution while using AWS Cloud Map in combination with systemd-resolved as my DNS resolver. Here's the situation:

1. I have a service registered with AWS Cloud Map in the local namespace.
2. The VPC and Route 53 configurations seem to be correctly set up, with DNS hostnames and DNS resolution options enabled for the VPC, and the Route 53 private hosted zone associated with the VPC.
3. However, my instance's /etc/resolv.conf file is using the default DNS resolver provided by systemd-resolved (127.0.0.53), and this setup is unable to resolve the hostname for the service discovery URL.

As a workaround, I updated the /etc/resolv.conf file to use the default VPC DNS resolver (169.254.169.253) as the nameserver, which allowed me to successfully resolve the hostname for my service discovery URL. However, I'm puzzled as to why the default systemd-resolved nameserver was not able to resolve the hostname in the first place.

Here are some relevant details:

• AWS Cloud Map Namespace:

  • ID: ns-tfdcofi4hm4x36vm
  • Type: DNS_PRIVATE
  • HostedZoneId: Z0644958TVA33WFXNTDW

• AWS Cloud Map Service:

  • Service ID: srv-47cvbl2ocflqrjxk
  • Instance ID: 6ed619db296a436981eea939efe1cb46
  • Attributes:
    • AVAILABILITY_ZONE: us-east-1a
    • AWS_INIT_HEALTH_STATUS: HEALTHY
    • AWS_INSTANCE_IPV4: 10.0.0.100
    • ECS_CLUSTER_NAME: sds-cluster
    • ECS_SERVICE_NAME: sds
    • ECS_TASK_DEFINITION_FAMILY: sds-family
    • REGION: us-east-1

• Route53 Record Set:

  • Name: local.
  • Type: A
  • TTL: 300
  • ResourceRecords: ["10.0.0.100"]

• Current /etc/resolv.conf:

nameserver 127.0.0.53
options edns0 trust-ad
search ec2.internal

In my previous project, I didn't encounter this issue and didn't have to modify the nameserver IP. Now, to access my backend hosted on ECS using sds.local, I had to change the nameserver, which I believe is not the ideal way.

• The project is inspired by https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-service-discovery.html

I would greatly appreciate any insights or guidance you could provide to help resolve this DNS resolution problem effectively. Thank you so much for your assistance!