Drawbacks of using custom DNS instead of route53 resolver in a VPC

What are the priorities of your organization? Every architectural decision is an exercise in prioritizing. ie. cost, performance, security, and resilience as well as nuances within each category. ie. do you need ultra-low-latency? or high QPS? what are your TTL requirements/priorities, ie. latency vs TTL? what are your automation requirements?

Route53 is purpose built to optimize for cost, performance, security, and resilience. It is the only AWS service with a 100% uptime SLA.

One little known feature of Route53 is that DNS queries are duplicated for redundancy. This is mentioned in How to achieve DNS high availability with Route 53 Resolver endpoints in the text between figures 2 & 3. This ensures optimal performance in the case where a DNS query is lost in-transit, since by default DNS uses UDP, a connection-less protocol. This prevents having to wait for a timeout and having to re-transmit the DNS query.

Route53 integrates with security services such as AWS DNS Firewall and Security Hub as well as observability services CloudTrail and CloudWatch. It also provides an API to automate record changes, ie. via Lambda in response to an EventBridge event or schedule. The integration of Route53 with Private Link is also non-trivial. For example, you can use Gateway endpoints for services, such as S3, to avoid incurring NAT GW costs for traffic from resources within VPCs to S3.

I hope this helps!

Whats the main drive to move to a custom DNS Server?

You've identified several important drawbacks of switching to custom DNS using a DHCP option set instead of using Route 53 Resolver. Let's expand on these and add a few more considerations:

1. VPC-local private hosted zones: You're correct that a spoke VPC would no longer be able to host and query its own VPC-local private hosted zone. This includes the inability to use VPC endpoints with private DNS.

2. Private Hosted Zones (PHZs): Indeed, PHZs would only be resolvable by setting up forwarding from your internal DNS to a Route 53 Resolver inbound endpoint.

3. AWS-specific features: The "EFS local AZ mount target" functionality you mentioned is one example of AWS-specific features that rely on Route 53 Resolver. Other examples include EC2 instance hostnames and certain AWS service endpoints that may not resolve correctly with custom DNS.

4. Resiliency: The built-in resiliency of Route 53 Resolver would be lost. You'd need to implement your own high-availability solution for your custom DNS servers. The failover behavior would indeed depend on the OS or application configuration.

5. DNS caching: Route 53 Resolver provides local DNS caching, which you'd lose with custom DNS.

Additional drawbacks and considerations:

6. Management overhead: You'd need to manage and maintain your own DNS infrastructure, which increases operational complexity.

7. Scalability: Route 53 Resolver automatically scales to handle DNS query volume, while custom DNS servers would require manual scaling.

8. Integration with AWS services: Some AWS services are designed to work seamlessly with Route 53 Resolver, and you might lose this tight integration.

9. DNS Firewall: If you're using or planning to use AWS DNS Firewall, it only works with Route 53 Resolver.

10. Hybrid cloud scenarios: Route 53 Resolver provides built-in features for hybrid cloud DNS resolution, which you'd need to implement manually with custom DNS.

11. Performance: Route 53 Resolver is optimized for AWS network topology, potentially offering better performance than custom DNS servers.

12. Cost: While there may be costs associated with Route 53 Resolver usage, managing your own DNS infrastructure could potentially be more expensive when considering maintenance and operational costs.

13. Compliance: Some compliance requirements might be easier to meet using AWS-managed DNS services rather than self-managed DNS.

While custom DNS via DHCP options can provide more control in certain scenarios, the drawbacks are significant, especially in terms of losing AWS-native functionality and increasing management overhead. Carefully consider your specific use case and requirements before making this switch.
Sources
What takes precedence: VPC's custom DHCP option set dns server or Route53 Resolver rules | AWS re:Post
AWS DNS firewall with custom DNS | AWS re:Post
Understanding Amazon DNS - Amazon Virtual Private Cloud
Route53 Private | AWS re:Post
Secondary DNS in an Amazon VPC - Hybrid Cloud DNS Options for Amazon VPC