I am encountering a persistent DNS resolution issue with an AWS Lambda function deployed within a custom VPC. The VPC is configured with a "private IPv4 / public IPv6 only" architecture, meaning instances and Lambda functions in the subnets have private IPv4 addresses and publicly routable IPv6 addresses.
My Lambda function is designed to resolve internal service names registered in AWS Cloud Map. The Cloud Map namespace is configured as a private DNS namespace within this VPC, and a generic "application-service" is defined within it, supporting both A (IPv4) and AAAA (IPv6) records.
The Lambda function is deployed into public subnets within this VPC and is associated with a security group that has broad egress rules (allowing all outbound IPv4 to 0.0.0.0/0 and all outbound IPv6 to ::/0). The Lambda's execution role has the necessary servicediscovery:DiscoverInstances permissions (probably not relevant for DNS resolution, but I was also trying to do it via the AWS SDK, with different issues).
Despite this configuration, the Lambda function consistently fails to resolve the Cloud Map service name. The Lambda's logs show repeated attempts to perform DNS lookups, specifically targeting an IPv4 DNS resolver at 169.254.78.1:53, and failing with a "no such host" error.
A critical requirement for this architecture is to avoid the costs associated with NAT Gateways and NAT64. We are specifically aiming for a solution that leverages the VPC or public IPv6 connectivity for outbound traffic and internal DNS resolution without incurring these additional charges.
My questions are:
- Why is the Lambda function attempting to use an IPv4 DNS resolver (
169.254.78.1:53) and failing, even though the VPC has an IPv6 DNS resolver (fd00:ec2::253) and the Lambda is in a public IPv6 subnet?
- In a "private IPv4 / public IPv6 only" VPC, what is the recommended best practice for configuring Lambda functions to reliably resolve Cloud Map service names, ensuring they leverage IPv6 for DNS resolution and explicitly avoiding NAT/NAT64?
- Are there specific VPC or subnet attributes (e.g., DNS64, or other routing configurations) that need to be explicitly enabled or configured for Lambdas in this architecture to ensure proper DNS resolution for both IPv4 and IPv6 records, without relying on NAT/NAT64?
- Could there be an implicit IPv4 dependency or routing issue within the Lambda's execution environment that prevents it from reaching the IPv4 DNS resolver, even if the security group allows it?
Any guidance on how to diagnose and resolve this DNS resolution failure in this specific VPC architecture, with a strong emphasis on avoiding NAT/NAT64 costs, would be greatly appreciated.
Thank you.
