GWLB: TG Sends traffic to unhealthy instances

0

I have the following scenario: 1 Security VPC and 1 Transit Gateway. Inside the security VPC, there are 3 AZs and 1 Gateway Load Balancer. In each AZ there is an endpoint GWLBE to redirect the traffic to the GWLB. The transit gateway attachment is configured in appliance mode. The target group of the load balancer has 3 instances: 1 VM series Palo Alto NGFW and 2 Linux machines to simulate unhealthy VM machines in each one of the AZs. The Palo Alto machine responds to the TCP 80 probes, while the other Linux machines time-out. The target group is configured with the new failover feature to redirect existing flows to healthy instances.

Testing: I am doing pings between 2 machines in 2 different VPCs. (I have also tested ssh traffic). I have found out that I have to send 2 to 4 times the ping command for the target machine to respond. After checking the flow logs, I found that the GWLB sends traffic to machines that are unhealthy, thus I only get the pings responses only when the GWLB send the traffic to the healthy Palo Alto. Why is the TG sending traffic to unhealthy machines?

2 Answers
0

Do you have Cross-Zone Load Balancing enabled?

--When to use Cross-Zone Load Balancing-- By default, the load balancer distributes traffic evenly across registered appliances within the same AZ. In this configuration, customers typically register more than one target within a single AZ behind the GWLB for firewall service availability and to distribute the traffic. In the event of a single target appliance failing health checks, the GWLB will route traffic to other healthy instances within the same AZ. This provides a cost-effective solution because the traffic does not cross AZ boundaries. While this setup is cost-effective, customers lose both the high availability and traffic distribution aspects in the event that all the targets in a specific AZ fail.

In order to achieve high availability and balanced traffic distribution, some customers choose another approach by enabling a feature called “cross-zone load balancing”. This feature makes it easier for you to deploy and manage your applications across multiple AZs. When you enable cross-zone load balancing, GWLB distributes traffic across all registered and healthy targets regardless of which AZs these targets are in. Enabling cross-zone load balancing incurs standard inter-AZ charges when the traffic crosses an AZ.

AWS
answered a year ago
  • I didn't. Somehow I think the Cross-Zone feature should be mentioned more in the documentation. It was hard to find.

    I tried it and now I see in the VPC flow of the 3 GWLB interfaces traffic to the Palo Alto. The problem now is that the default gateway of the Palo Alto is the GWLB interface of the AZ 1. How do I configure the Palo Alto so it forwards back the packet to where originally came from?

0

How do I configure the Palo Alto so it forwards back the packet to where originally came from?

  1. you will need to establish full mesh geneve tunnels between gwlb eni in 3 AZs and palo alto data interface
  2. configure specific subnets routes for subnets in those 3 AZs, each next hop will need to be the gateway IP of that subnets

do those steps will help you forwards back the packet to where originally came from

answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions