How to troubleshoot connectivity issues between AWS VPC and on-premises via AWS Direct Connect Transit VIF

Step 1: Check if the source instance is allowing the traffic towards destination IP in SECURITY GROUP and NACLs.

Step 2: Check if the source instance’s subnet has a route for the destination IP via TGW [x.x.x.x/x → tgw-abcd].

Step 3: Check the outbound NACL associated with the Source EC2 and confirm that it has necessary outbound rule to allow traffic to destination IP and port.

Step 4: Check if the customer has the same AZ/Subnet enabled for an attachment where the source Instance is present.

Step 5: If that’s correct, check the NACLs associated to the source side of the VPC TGW attachment ENI’s subnet. If NACLs is restricted it should allow traffic from Source Instance IP.**

Step 6: Check if the source TGW VPC attachment route table has a route for the destination IP pointing towards DXGW attachment. Ensure the correct route is being propagated to the TGW route table associated with the source VPC attachment.**

Step 7: Confirm if the correct routes are being advertised from the on-premise CGW device.

Step 8: If all looks good, perform a Bi-directional traceroute to know if the** traffic is reaching Amazon Peer IP and Customer Peer IP.

Step 9: Check the Packets Out cloudwatch metric for the DXGW attachment and also if "PacketDropCountBlackhole" or "PacketDropCountNoRoute“ count is increasing.

Step 10: If everything looks good as per the previous steps and the connectivity issue persists, enable VPC flow log on TGW ENIs, Source and Destination ENI using Flow Log Format as ${action} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${pkt-srcaddr} ${pkt-dstaddr}. Check the flow logs on the TGW ENI to see if the traffic is seen and not being REJECTED.

Step 11: Enable TGW Flow logs on the Transit Gateway and check if the flow is seen on source and destination attachments. Look for 'packets-lost-blackhole', 'packets-lost-no-route' and 'packets-lost-mtu-exceeded' fields.

Additonal Tips:

• VPC Reachability Analyzer tool can used to check the connectivity between the source and the destination on any particular port.

References:

[+] Transit Gateway Flow logs: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.html

[+] Public Docs for NACLs: https://docs.aws.amazon.com/vpc/latest/tgw/tgw-nacls.html

[+] Network Manager Route Analyzer: https://docs.aws.amazon.com/network-manager/latest/tgwnm/route-analyzer.html

[+] VPC Reachability Analyzer: https://docs.aws.amazon.com/vpc/latest/reachability/how-reachability-analyzer-works.html#source-and-destination-resources

[+] Cross-account analyses for Reachability Analyzer: https://docs.aws.amazon.com/vpc/latest/reachability/multi-account.html

[+] TGW Service-Linked Role: https://docs.aws.amazon.com/vpc/latest/tgw/service-linked-roles.html