Any Policy route on AWS for Site to Site VPN Migrate to Direct Connect, in phases?
We're currently doing Site to Site VPN connection from On-Prem to AWS, where the VPN attachment to the Transit Gateway. We have plan to setup a Direct Connect to AWS. I have a doubt for the migration from S2S VPN to Direct Connect and i need AWS Expertise to advise. Thanks in advance.
A little Background: We have Site-to-Site VPN connections from our on-prem to AWS. Currently doing BGP route advertisement between On-Prem and AWS.
What we planned: New Direct Connect link to AWS. Phase to Phase migrate AWS hosting Applications to use Direct Connect. Priority to Critical AWS Apps that required dedicated SLA network connections.
My doubt: The Current Site to Site VPN is via Internet Firewalls. The new Direct Access link will be positioned over another set of WAN Firewalls. From network perspective, On-Prem i can easily route the Dst IP based on AWS Apps to be migrated to Direct Connect in phases. However, the On-Prem Source IP (the users, the monitoring system) are the same to multiple AWS Apps. From AWS transit gateway, Is there a policy-route we can do to route to Direct Access link base on SRC & DST IP? If not, I foresee such communication will be breaking over the different set of Firewall due to the asymmetric routing. (The return packets from AWS to the on-prem source IP). Can I get any advise on this area? Thanks again!
- Newest
- Most votes
- Most comments
Your right about the asymmetric routing.
I think the only way would be to migrate a VPC at a time and have a seperate Transit Gateway Route table. One for the VPN and the other for the DX. Depending on how your applications are seperated, i.e. Each app has its own VPC then this would work. If your APP shares the same Subnet/CIDR Range then I dont think theres a way to migrate a subnet..
Thanks for sharing, Gary. The idea of having different routing tables attached to different VPCs seems promising, but I'm still encountering challenges with this approach. Let me provide more context: We're currently utilizing AWS Central Control Tower Architecture design. In this setup, all Applications VPCs are utilizing a single Infra-Routing table. This table directs traffic through the Centralized Firewall VPC for packet inspection to and from the On-premises network. Here's where the challenge lies: If I were to implement different routing tables for the migrated VPCs to be routed via Direct Connect (DX), I could potentially bypass the AWS firewall—something that's not feasible. Alternatively, if I create different routing tables and set them to route traffic back to the Centralized AWS firewall, I'd essentially end up where I started. The same routing table on the Inspection Firewall wouldn't be able to policy-route the traffic to the On-premises network. Together with the different routing tables idea, i could only think the possible of having a new AWS Centralize FW which dedicates to the path towards DX to on-prem? So that we can have a clean path with same set of firewalls via the DX?
Yes I see your challenge now. I’ll have a think. But yes your approach would work with a new set of FW in a new VPC
Thanks Gary, appreciated your Expert response. Let's open this thread and see if i could have other AWS expertise provide different or better opinion.
Relevant content
- asked 3 months ago
- asked 8 months ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated a year ago
Hi Gary, thanks for your answer. Migrate VPC 1 after another is not an issue, the issue is the return packets that route asymmetric via different Firewall.
Let me elaborate further with sample. Let's imagine my on-prem user sitting in 10.1.1.0/24. They're current accessing AWS VPC 1 with CIDR 10.10.10.0/24 and VPC 2 with CIDR 10.10.20.0/24. I can easily route from on-prem 10.10.10.0/24 to new DX link(via WAN FW)) and retain 10.10.20.0/24 via S2S VPN(via INET FW). But how about the return path? If i can't do policy based routing, AWS can only route 10.1.1.0/24 return either to S2S VPN or EX. In any case, partial of the return packets will drop at Firewall since there's no stateful session record on it. I need a solution to tackle this situation.
As your using transit Gateway, you would want different routing tables attached to different VPC's. Have a VPN Transit Gateway Routing table (A for this case) and have a direct Connect routing table (B for this case) The VPN would terminate in the TGW Route Table A and DX to Route Table B. As you migrate the Apps over, Attach the correct route table to the correct VPC and internally update your route tables to Route VPC CIDRs to either VPN or DX