My AWS NLB on ca-central-1 started having issues since 7am UTC on September 6th 2024. No changes were made and there are several other regions configured in the same way while only this one is having this issue.
The NLB sees all targets as healthy. The NLB is configured across 3 AZ's and has Cross-zone load balancing set to off. All targets are in a single AZ. It has always been configured like this since 2022 and worked fine until today in this specific region. Since this morning, requests started failing intermittently and it has been confirmed that it is only happening when the traffic is received on one of the NLB addresses that is not in the same AZ as where all the targets are. ie traffic is received in NLB IP address from AZ b or AZ c, but the node where the traffic needs to go to is in AZ a. What is seen in other regions with the same setup is that only 1 IP is returned on the DNS query for the NLB DNS name, while for ca-central-1, since this morning, all 3 IP addresses from all AZ's are returned. The intermittency goes away when Cross-zone load balancing is set to on.
There are no events reported in the AWS health status page and I have found no communication about such a change in NLB service behaviour.
Is anyone aware of AWS planning to make such a change in NLB behavior where all NLB IP addresses from all of its AZ's would be announced despite all of its targets being in a single AZ and Cross-zone load balancing being set to off, in contrast to the behaviour so far, in which only the IP address of the NLB from the same AZ in which all the targets are is announced?
Or would this be an AWS service incident?
AWS OFFICIALUpdated 20 days ago
