How do I troubleshoot the Gateway Load Balancer’s connectivity issues?

Resolution

Prerequisites

Before you begin, complete the following tasks:

• Check the network's architecture and all its intermediate devices to verify that they meet the connectivity requirements.
• Confirm that the security groups, network access control lists (network ACLs), and the route tables are correctly configured.
• Confirm that the Amazon Virtual Private Cloud (VPC) endpoint for the Gateway Load Balancer is available.
• Verify that there are healthy appliances behind the Gateway Load Balancer. If cross-zone isn't activated, there's at least one active, healthy target in each Availability Zone.
• If you have an AWS Transit Gateway, activate the appliance mode on the Transit Gateway attachment that's associated with your hosting network's VPC.

Troubleshooting tips

If you continue to get connectivity errors, use the following troubleshooting tips to establish a connection.

Set up the VPC Flow logs and packet captures for analysis

Activate the VPC Flow logs on the network interface of the Gateway Load Balancer's endpoint. Use these additional fields during setup:

${pkt-srcaddr}, ${pkt-dstaddr}, ${flow-direction} and ${tcp-flags}

Get simultaneous packet captures from the client, the server, and the target appliances behind the Gateway Load Balancer.

Use either Wireshark or tcpdump to capture packets. Then, use Wireshark to analyze the packets. For instructions on how to use Wireshark, see the Wireshark website.

Troubleshoot the forward traffic flow

Example of a forward traffic flow:

Client--->GWLBe--->GWLB--->Appliance--->GWLB--->GWLBe--->Server

Use a tool such as Wireshark to open the client's packet capture to see if the packets leave towards the server's IP address.

Check the VPC Flow logs of the network interface on the Gateway Load Balancer's endpoint. Use these logs to verify if the packets reach the network interface.

Open the packet capture for the Gateway Load Balancer's target appliance. Verify if the packets enter the target appliance and exit towards the Gateway Load Balancer.

Check the VPC Flow logs of the network interface on the Gateway Load Balancer's endpoint. Use these logs to verify if the packets exit the network interface toward the server.

Check the server's packet capture to see if the packet reaches the server.

Troubleshoot the reverse traffic flow

Example of a reverse traffic flow:

Server--->GWLBe--->GWLB--->Appliance--->GWLB--->GWLBe--->Client

Check the server's packet capture to see if the server generates a response.

Check the VPC Flow logs of the network interface on the Gateway Load Balancer's endpoint. Use these logs to verify if the response packet reaches the endpoint. Make sure that it's the same endpoint that managed the forward flow.

Open the packet capture for the Gateway Load Balancer's target appliance. Check if the response packet enters the appliance and exits towards the Gateway Load Balancer.

Check the VPC Flow logs of the network interface on the Gateway Load Balancer's endpoint. Use these logs to see if the packets exit the network interface towards the client.

Use packet capture tools like Wireshark to verify that the response packet reaches the client's network interface.

Notes:

VPC Flow logs don't log the communication between the Gateway Load Balancer's network interface and its endpoint's network interface.

The packets between the Gateway Load Balancer and the appliances are GENEVE-encapsulated over UDP. For this reason, you can't use the load balancer's or the target's network interface VPC Flow logs to trace client or server traffic.

Note the source port and the time when the failed TCP connection was initiated. Use the source port and time to trace and follow the connection through subsequent devices along the network path.