How do I troubleshoot and resolve cross-account connectivity issues through my transit gateway?

Resolution

Note: The account that owns the transit gateway is the owner account. The account that receives access to the transit gateway is the shared account.

Check your transit gateway resource sharing settings

Note: If you have an organization in AWS Organizations, then turn on resource sharing. When you turn on resource sharing, AWS automatically shares and accepts transit gateways between member accounts.

Complete the following steps:

1. Sign in to the AWS Resource Access Manager (AWS RAM) console with the owner account.
2. In the navigation pane, choose Shared resources.
3. Select your transit gateway.
4. Check whether you shared the transit gateway with the correct account or organization.
5. Verify that Share status is Associated and doesn't show Pending acceptance.

Check the transit gateway attachment status in the owner and shared accounts

Complete the following steps:

1. Open the Amazon Virtual Private Cloud (Amazon VPC) console.
2. In both the owner and shared accounts, choose Transit Gateway Attachments in the navigation pane.
3. Verify that the Attachment state is Available.
   Note: Each account that connects to the transit gateway requires an attachment that's in an Available state.
4. Confirm that the attachment uses one subnet from each Availability Zone for traffic routing.

If the attachment shows Pending acceptance in the shared account, then complete the following steps:

1. From the shared account, choose Transit gateway attachments.
2. Select the pending attachment.
3. Choose Actions, and then choose Accept.

Check route table settings and route propagation

Complete the following steps:

1. Sign in to the AWS Transit Gateway console with the owner account.
2. In the navigation pane, choose Transit gateway route tables.
3. Check whether each attachment associates with the correct route table.
4. Verify that your route table contains routes to the CIDR blocks of other VPCs. The routes can be either static or propagated, and they must point to the correct transit gateway attachment for each VPC.
5. (Optional) If you use a security VPC to inspect traffic, then confirm that firewalls and security appliances allow traffic.
6. Confirm that source and destination ranges don't overlap.
   Note: A transit gateway can't route overlapping IP address ranges.

It's a best practice to create segmented route tables for your environments. For example, if you have developer and production environments, then segmented route tables isolate traffic between them because each environment has its own route table.

Verify that VPC route tables point to the transit gateway

Complete the following steps:

1. Open the Amazon VPC console.
2. In the navigation pane, choose Route Tables.
3. Choose the route table that's associated with your resource's subnets.
4. Under the Routes tab, verify that routes to the CIDR block of other VPCs point to the correct transit gateway ID.

Note: Confirm that your security groups and network access control lists (network ACLs) allow traffic between your VPCs.

Verify security group and network ACL configurations

Take the following actions:

• Check whether Amazon Elastic Compute Cloud (Amazon EC2) security groups allow inbound and outbound traffic for the correct CIDR blocks and ports.
• Configure your security groups to allow the required protocols between your workloads. For example, check TCP port 443 and TCP port 22.
• Check whether network ACLs allow traffic flow between VPCs.
• Check your network ACL rules and verify that they don't block traffic to the transit gateway.

Note: Network ACLs are stateless. You must allow both inbound and outbound traffic in your network ACL rules.

Analyze paths with Reachability Analyzer

Note: To use VPC Reachability Analyzer and analyze paths between accounts, turn on trusted access in AWS Organizations.

Complete the following steps:

1. Open the AWS Network Manager console.
2. In the navigation pane, choose Network Manager.
3. Choose Reachability Analyzer.
4. Choose Create and analyze path.
5. Enter the following information to choose the path source and path destination:
   For Source account, select the source account's account ID.
   For Source type, select the resource type.
   For Source, select the specific resource.
   For Destination account, select the destination account's account ID.
   For Destination type, select the resource type.
   For Source, choose the specific resource.
6. Choose Analyze path.
7. Review the results.

If the path is Reachable, then your network configuration is correct. If the path is Not reachable, then modify your routes and security rules to allow traffic.

Check traffic flows with VPC Flow Logs

Before you begin, create a VPC flow log. It's a best practice to use a custom format that includes the pkt-srcaddr and pkt-dstaddr fields. The pkt-srcaddr and pkt-dstaddr fields in VPC flow logs show traffic with the original source and destination host IP addresses. The default srcaddr and dstaddr show intermediate network interface IP addresses.

Use VPC Flow Logs to identify accepted and rejected traffic between VPCs that connect through your transit gateway.

To analyze flow logs, complete the following steps:

1. Open the Amazon CloudWatch console.

2. In the navigation pane, choose Logs, and then choose Log Insights.

3. From the Query Scope dropdown list, select the log group for your VPC flow log. Then, search the elastic network interface ID for the source and destination resource.

4. Run the following example query to filter logs by source and destination IP addresses to identify accepted and rejected traffic between VPCs:

   parse @message " *************************" as version, accountid, interfaceid, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, logstatus, vpcid, subnetid, instanceid, tcpflags, type, pktsrcaddr, pktdstaddr, pktsrcawsservice, pktdstawsservice, flowdirection, trafficpath|filter (pktsrcaddr='SOURCE-IP-ADDRESS’ and pktdstaddr=‘DESTINATION-IP-ADDRESS’)|limit 100

   Note: Replace the ************************ and the field names to match the format that you selected when you created the VPC flow logs. The number of asterisks and fields varies based on your chosen log format. Replace SOURCE-IP-ADDRESS with your source IP address and DESTINATION-IP-ADDRESS with your destination IP address.

Test connectivity between Amazon EC2 instances

Run the following commands to perform connectivity tests between your EC2 instances that connect through the transit gateway.

Note: In the following commands, replace DESTINATION-PRIVATE-IP with your destination's private IP address and PORT with the port number that you want to test.

Run the following ICMP ping command:

ping DESTINATION-PRIVATE-IP

Run the following telnet command to test TCP ports:

telnet DESTINATION-PRIVATE-IP PORT

Run the following curl command to perform an HTTP test:

curl -v http://DESTINATION_PRIVATE_IP:PORT

If the tests fail, then take the following actions:

• Verify that the route tables in both VPCs contain entries that point to the transit gateway for destination CIDR block ranges.
• Verify that the transit gateway route table contains entries for both VPCs and associates with the correct transit gateway attachments.
• Verify that security groups allow inbound and outbound traffic between instances.
• Verify that network ACLs in your subnets allow the required traffic.
• If you have a shared transit gateway, then verify that you shared it correctly with the target account.

Related information

How do I share my transit gateway with another account or within an organization?

How Reachability Analyzer works

Transit gateway route tables in AWS Transit Gateway