Direct connect active to specific region

Question

Dear Team - This is my first question. Hope, i will get the details i needed. i am working on similar deployment as mentioned on scenario-1 / Figure-3 on https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/. However, i wanted to know how we can save the cost to avoid backup traffic through TGW peering. for that, i am thinking below.

1 - Advertise both 172.16.0.0/16 and 172.17.0.0/16 from both on-prem router

2 - On the PHX transit VIF, set 7224:7300 for 172.16.0.0/16 and 7224:7200 for 172.17.0.0/16.

3 - On the ATL transit VIF, set 7224:7300 for 172.17.0.0/16 and 7224:7200 for 172.16.0.0/16.

Note: Above is as per https://repost.aws/knowledge-center/direct-connect-bgp-communities

4 - On the Direct Connect Gateway where both the TVIF are connected, add 10.0.0.0/16 and 10.1.0.0/16 in the allowed prefix.

Once, i configure this, i am expecting below behavior

In normal situation, where all is up, connectivity from 172.17.0.0/16 --> 10.1.0.0/16 must take ATL <--> US-east-1
In normal situation, where all is up, connectivity from 172.16.0.0/16 --> 10.0.0.0/16 must take PHX <--> US-west-1
In case of PHX direct connect down, ATL on-prem can still connect US-WEST-1 VPC.
In case of PHX direct connect down, we should be able to ping PHX 172.16.0.0/16 resource through

US-WEST-1 --> Direct connect Gateway --> ATL --> MPLS --> 172.16.0.0/16(PHX resource)

same should be the behavior in case of ATL is down.

Let me know if you see any issue in the above understanding ? or it is not supported scenario and the only option is to go with LMP

Thanks, JD.

Accepted Answer

Thanks a lot Azeem for your reply. Deserved my vote :). Really helpful. Before i finalize the design, my other network engr wanted to check why the below design can not work with single DXGW ? because as per the TGW route table, there are two entries for both 172.16 and 172.17 as we are advertising both the CIDR range (172.16 and 172.17) from both the location. I had a doubt if this is possible. can you also share your thought. In this case also, can we achieve the same thing as two DXGW mentioned in your design? this will help simplifying the design.

Enter image description here

![Enter image description here](/media/postImages/original/IM3ZdPg9wcQuC84UJGByrB1A)

Answer

Thanks for your question JD.

In your case which is based on scenario 2 of the [blog](https://aws.amazon.com/blogs/networking-and-content-delivery/influencing-traffic-over-hybrid-networks-using-longest-prefix-match/). With your configs (no static route on TGW peering for on-prem prefixes), you will be running without resilliency.
TGW us-west-1 will have 172.16.0.0/16 (7224:7300) and 172.17.0.0/16 (7224:7200). PHX On-prem will have 10.0.0.0/16 and 10.1.0.0/16. And similary on TGW us-east-1 will have  172.17.0.0/16 (7224:7300) and 172.16.0.0/16 (7224:7200). ATL On-prem will have 10.0.0.0/16 and 10.1.0.0/16.

When every thing is up traffic will be fine, but you will not have resiliency.

Scenario 1: In case DX Connection in PHX fails.
Assuming you gave iBGP between sites over the corporate and advertise 10.0.0.0/16 from ATL to PHX on the MPLS.

Traffic from PHX to us-west-1 will flow like this:
PHX --> MPLS --> ATL --> DXGW(65002) --> TGW(us-east-1) --> TGW(us-west-1)

Return traffic from TGW (us-west-1) to PHX will be dropped since the TGW in us-east-1 does not have any route for 172.16.0.0/16.

Instead I would suggest use the appoach I share here in a rough drawing. Since we have increased the [limit](https://docs.aws.amazon.com/directconnect/latest/UserGuide/limits.html) on number of transit VIF, you could leverage BGP in a better manner to achieve resilient and operationally excellent design and save cost of inter-region Data Transfer over tranit gateway peering.
**You will need to use correct BGP metric on-prem to ensure symetric traffic flow such as Local-Pref.**

In the diagram I show **route advertisement from on-prem to AWS** which will **influence** the **traffic path from AWS to on-prem**

![Using multiple Transit VIF on same DX Connection](/media/postImages/original/IM7doir8koTBSKuECsWnLevA)

Direct connect active to specific region

Relevant content