Routing VPC to VPC traffic through an on-prem firewall via Transit Gateway

Question

A customer is trying to setup VPC to VPC routing through their on-prem firewall over TGW.  The desired behavior is that traffic from VPC-A will route through the on-prem firewall to get to VPC-B.  With the current setup, the traffic routes from VPC-A to VPC-B without making it to the on-prem firewall.  When we perform a traceroute, the 2nd hop in the path is a 169.254.x.x address, which I believe may be the DXGW or something similar.  I can replicate the same behavior If I have a 0.0.0.0 route defined to a nat gateway as well, but in that case the 2nd hop is the nat gateway address.  The customer POC setup is as follows:

VPC-A - 10.0.0.0/24  
VPC-B - 10.0.1.0/24  
DXGW connected to DX via TVIF

**VPC-A-Route-Table Routes**  
10.0.0.0/24, local  
0.0.0.0, TGW

**VPC-B-Route-Table Routes**  
10.0.1.0/24, local  
0.0.0.0, TGW

**TGW -VPC-Traffic route table**  
**associations:**  
VPC-A  
VPC-B

**Propagations:**  
DXGW

**Routes:**  
On-Prem routes, propagated  
0.0.0.0/0, DXGW attachment, static

**TGW - On-prem traffic route table**
**Associations:**  
DXGW

**Propagations:**  
VPC-A  
VPC-B

**Routes:**  
10.0.0.0/24  
10.0.1.0/24

I believe we are missing an explicit route to tell traffic to use the on-prem firewall for routing of VPC to VPC traffic, but I am not exactly sure of the best place to configure that in this scenario.

Accepted Answer

The problem you are hitting is at DXGW. Since same DXGW is being used for egress/ingress path. DXGW can not allow you to route traffic for destination that is advertised from AWS side.

You need to separate the egress path and ingress path. Possible options are 1) two separate DXGW paths, 2) one DXGW (egress) with one VPN(ingress) or vice versa, 3) two VPN connections (egress, ingress).

At the same time I would advise customer to not do it as it is suboptimal routing, creates delay, dependency for intra region traffic on external connectivity. It is recommended to do that inspection via Inspection VPC which inspects the inter VPC traffic. You can use AWS Network Firewall or 3rd party firewalls in the inspection VPC.

Routing VPC to VPC traffic through an on-prem firewall via Transit Gateway

Relevant questions

How do I set up Amazon VPC ingress routing with a stateless network appliance?

Routing VPC to VPC traffic through an on-prem firewall via Transit Gateway

Security VPC is not working with Transit Gateway

Routing network traffic between two EC2 instances in the same subnet to a firewall appliance in another VPC

VPC peering and Network Firewall

Data Transfer OUT Charges Through TGW in Another Account

DNS Query from ec2 instance not hitting Palo alto firewall

Transit Gateway/DX route propagation to VPC route tables

Transit Gateway and AWS Network Firewall

Transit Gateway attachment cost to VPC and subnets