Overlapped PHZ with multiple VPCs

Question

I'm using this aws architecture blog post for reference - https://aws.amazon.com/blogs/architecture/using-route-53-private-hosted-zones-for-cross-account-multi-region-architectures/ .  For now, I'm using multiple VPCs in a sandbox account, so multi-account is not a factor.  There is no hybrid dns, this is purely within AWS.

I have a PHZ, name.local, associated with the "network" vpc, an overlapped PHZ, dev.name.local, associated with the "dev" vpc and the "network" vpc.  I've created an A record "test" in the corresponding PHZ (so test.name.local and test.dev.name.local).  I've spun up an amazon linux ec2 instance in both vpcs.  From the ec2 instance in the "network" vpc, I can resolve both test.name.local and test.dev.name.local.  From the ec2 instance in the "dev" vpc, I can resolve test.dev.name.local, but test.name.local will not resolve.

The blog says "For a consistent view of DNS and efficient DNS query routing between the AWS accounts and on-premises, best practice is to associate all the PHZs to the Networking Account."  This is what I've done, but DNS is not consistent between the VPCs.   I did exclude components 2 (resolver endpoints for on-premises integration) and 3 (resolver rules) mentioned in the blog post since the description says they are for hybrid dns.

Why isn't this working?

Answer

The difference in that blog post is that there is resolution of the "root" DNS name set up in the Application accounts. If someone wants to resolve "aws.customer.local" from within one of those accounts then there are forwarding rules in there for "customer.local" to be resolved on-prem, and then forwarding rules on-prem to resolve "aws.customer.local" in the Networking account.

Your setup currently doesn't have anything allowing resolution of "name.local" from your "dev" vpc. The simplest solution is probably to associate the name.local PHZ with the "dev" VPC.

Your setup currently doesn't have anything allowing resolution of "name.local" from your "dev" vpc.  The simplest solution is probably to associate the name.local PHZ with the "dev" VPC.

Answer

Looking at this with a fresh set of eyes, I noticed a consideration at the bottom of the blog post - "If Application 1 needs to communicate to Application 2, then the PHZ from Account A must be shared with Account B. DNS queries can then be routed efficiently for those VPCs in different accounts."  This is what I'm trying to do.

This led met to find this statement in another of their blog posts - "When a Route 53 private hosted zone needs to be resolved in multiple VPCs and AWS accounts as described earlier, the most reliable pattern is to share the private hosted zone between accounts and associate it to each VPC that needs it."

"associate it to each VPC that needs it" - in a cross account scenario, you must create an authorization (manual step, CLI or API) from one account and then an association (also manual, CLI or API) from the other account.  So If you have five accounts/vpcs with their own overlapped PHZ that need to communicate, that's twenty auths and twenty associations you have to setup.  Add one more account in the future, that's 10 more auths and 10 more assoc you need to add.  You want to do VPC endpoints in a shared services account?  that's even more PHZs you need to deal with.

This feels like a significant gap in functionality compared to how Transit Gateway provides routing between a large number of accounts.  That routing is going to be of limited usefulness if you don't have a DNS solution to go with it, that is just as scalable as the transit gateway.

Overlapped PHZ with multiple VPCs

Relevant content