NLB stops responding on one IP

Question

Hi everyone,

We've seen this happen a couple of times now. I was wondering if anyone has insights on why this happens, and/or what we can do to prevent or fix it.

Basically, One of the IPs in the network loadbalancer stops responding. We check to make sure the IPs didn't change in DNS or there was another DNS caching issue. I can see no errors or warnings in the target group settings or NLB settings....

$ dig NV-SFTP-LB-489fe716f98a2088.elb.us-east-1.amazonaws.com @ns-1286.awsdns-32.org.

; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> NV-SFTP-LB-489fe716f98a2088.elb.us-east-1.amazonaws.com @ns-1286.awsdns-32.org.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50240
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;NV-SFTP-LB-489fe716f98a2088.elb.us-east-1.amazonaws.com. IN A

;; ANSWER SECTION:
NV-SFTP-LB-489fe716f98a2088.elb.us-east-1.amazonaws.com. 60 IN A 172.16.1.85
NV-SFTP-LB-489fe716f98a2088.elb.us-east-1.amazonaws.com. 60 IN A 172.16.0.212

But, 172.16.1.85 is not responding while 172.16.0.212 does:

$ telnet 172.16.1.85 22
Trying 172.16.1.85...
telnet: Unable to connect to remote host: Connection timed out

$ telnet 172.16.0.212 22
Trying 172.16.0.212...
Connected to 172.16.0.212.
Escape character is '^]'.
SSH-2.0-srtSSHServer_11.00

Edited by: dh42 on Feb 15, 2021 9:36 AM

Edited by: dh42 on Feb 15, 2021 12:08 PM

Accepted Answer

Hello,  
  
As you are not able to connect to one of the NLB IP for this NLB, I would request you check that there should be backend target present in the same AZ as of this NLB node then only the connection will be successful.  
If you only have one target then make sure that NLB attribute cross-zone is enable for this NLB, so that both node can connect to the same target. If there is target present for the NLB node than only we can connect to that NLB node.  
  
\[] Network Load Balancers - Availability Zones - https://docs.aws.amazon.com/elasticloadbalancing/latest/network/network-load-balancers.html#availability-zones  
  
"After you enable an Availability Zone, the load balancer starts routing requests to the registered targets in that Availability Zone. Your load balancer is most effective if you ensure that each enabled Availability Zone has at least one registered target."  
  
Thanks

Answer

Thanks.  For this particular scenario, we have 1 target behind the NLB, but the NLB is provisioned to multiple AZs.   Enabling the cross-zone feature on the NLB allows this to work again.

NLB stops responding on one IP

Relevant content