Why VPC PrivateLink as ALB target_group member always comings as unhealthy, where as reachability analyzer ac reach the target?
Hi there,
I'm keep getting hit by this time to time without a proper solution.
I have an ALB in account-A, wich is forwarding the traffic to target-group, which has the IPs of the VPC PrivateLink as the member, which is paired up with a VPC EndPoint Service on Account-B and the pair of EC2 (running Nginx) as the member of the associated NLB. When I use the cross account reachability analyzer, it reaches the Nginx instances (on Account-B) successfully.
But in reality, the ALB target-group (on Account-A) always has the unhealthy target:
Hence, I'm getting 502, trying access the service running on Nginx server. Last time when it happened, it was suddenly fixed by itself but I made some learing out of it:
- AWS doesn't like if the number of used AZs on the EndPoint Service account (e.g. account-B) doesn't match with the number of AZs on the VPC PrivateLink account (e.g. account-A)
- The NLB that associated with the the EndPoint Service (on account-B) needs to be cross-zone load balancing enabled
- Also, I read somewhere that at least two registered targets are required (on account-B behind the NLB) to make it work but not very sure about it.
This time I made sure that all of the above are there but still got the unhealthy targets. Can anyone help me debugging this issue pls?
-S
- Newest
- Most votes
- Most comments
Hi there,
What health checks have been configured on the ALB could you share this?
First of all check your security groups and NACLs that they are allowing traffic. verify that path / is appropriate for the application running behind the ALB Ensure that the target group's protocol and port settings match the protocol and port on which the application is running. Ensure that the target instances behind the ALB are in healthy state. Check the ALB's listeners to make sure that they are correctly configured to route traffic to the target group. verify that the ALB's SSL/TLS settings match the protocol and settings of the target instances. If necessary, enable access logs for the ALB to get more insights into the traffic flow and any potential issues.
for a more detailed methodology see https://repost.aws/knowledge-center/elb-fix-failing-health-checks-alb
Health checks and traffic ports on target groups have different settings.
Please ensure your health check configs have correct protocol http or https and port.
Please can you share your security group for your registered targets.
Also the Instance IP address in your reachabilty screen shot is different than the ones registered in the target group. Is that correct?
Please find the screen-shots added below
Hi, please find the health-check setting:
Below is the security-group setting that attached to the VPCE PrivateLink:
re. IP address mismatch, I have two target groups, pointing to two VPCE, with identical configuration. I may have ran the analyzer on one and paster the screenshot of the other.
-S
Relevant content
- Accepted Answerasked a year ago
- asked a year ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
One thing I think I should have mentioned that the listener for that ALB is HTTP, which is forwarding the traffic, based on host-header, to the HTTPS target group - is that causing the issue? Rest of the thing are all good. The SGs and NACLs are okay, otherwise reachability analyzer wouldn't succeed. The applications (EC2 Nginx) behind the NLB are healthy and matches with port and path as the target group health-check configuration. If I change the target group and the applications to listen on port 80, it works okay but target group seems to become unhealthy when HTTP listener is forwarding the traffic to HTTPS target group.
I been through that troubleshooting page already, which didn't shade any light at all and also found the information provided s very generic.