Split-horizon DNS for mixed Databricks connectivity
My customer have ec2 instances in particular VPC of their account. They want to connect to databricks from ec2 instances. They want to connect to 2 different workspace, A and B.
'A' is available over public internet on host xyz.cloud.example.com. 'B' is available over private link on host abc.cloud.example.com.
For 'B', they have created privatelink connection between their VPC and databricks. Also created private dns zone 'cloud.example.com' attached to VPC. Added record in it abc.cloud.example.com pointing to privatelink endpoint. B connection works properly after this setup. But for A, it stops working because ec2 instance tries to resolve DNS xyz.cloud.example.com on private zone. How can customer solve this usecase, they need to have connection to both private & public endpoints of databricks. Since base domain is same (cloud.example.com), customer is not able to resolve public DNS records. Is there a way where customer can tell to use private zone only for specific set for DNS records.
- Tags
- Amazon Route 53Amazon VPCDomain Name System (DNS)Amazon Route 53 Resolver DNS FirewallNetworking & Content Delivery
- Language
- English
asked 10 months ago142 views
- Newest
- Most votes
- Most comments
This is a classic split-horizon DNS scenario where you need different DNS resolution behavior for the same domain depending on the context. Here's how to solve this issue:
You can implement split-view DNS (also known as split-horizon DNS) in Amazon Route 53 to handle both public and private resolution for the same domain. For your specific case:
-
Create both public and private hosted zones with the same name (cloud.example.com).
-
In the private hosted zone (which is already associated with your VPC):
- Keep the record for abc.cloud.example.com pointing to your PrivateLink endpoint
- In the public hosted zone:
- Create a record for xyz.cloud.example.com pointing to the public Databricks workspace A
- For the specific record that needs public resolution (xyz.cloud.example.com), you need to create a specific forwarding rule using Route 53 Resolver:
- Set up an outbound Route 53 Resolver endpoint in your VPC
- Create a forwarding rule for xyz.cloud.example.com that directs queries to public DNS resolvers
- This will override the private hosted zone for this specific record
This configuration will allow your EC2 instances to resolve abc.cloud.example.com to the private PrivateLink endpoint while still resolving xyz.cloud.example.com to the public IP address.
Alternatively, you could use Route 53 Resolver rules to implement conditional forwarding, where queries for specific records within your domain are forwarded to different DNS resolvers based on your requirements.
With this setup, your EC2 instances will be able to connect to both Databricks workspaces - workspace B through the PrivateLink connection and workspace A over the public internet.
Sources
Considerations when working with a private hosted zone - Amazon Route 53
Streamline hybrid DNS management using Amazon Route 53 Resolver endpoints delegation | Networking & Content Delivery
Route 53 Resolver and on-premises DNS servers | AWS re:Post
AWS PrivateLink concepts - Amazon Virtual Private Cloud
answered 10 months ago
Relevant content
asked a year ago
