Skip to content

Auto Scaling Group does not replace instance when one of multiple Target Groups reports Unhealthy

0

Hello,

I have an Auto Scaling Group (ASG) with the health check type set to ELB, and it is attached to two Target Groups (TG1 and TG2). One EC2 instance in the ASG is registered with both Target Groups.

  • In TG1, the instance is marked as Healthy
  • In TG2, the same instance is marked as Unhealthy

However, the instance still shows as Healthy in the ASG, and Auto Scaling does not terminate or replace it.

According to the documentation (https://docs.aws.amazon.com/autoscaling/ec2/userguide/attach-load-balancer-asg.html):
"Amazon EC2 Auto Scaling can replace a running instance if any of the Elastic Load Balancing load balancers you attach to the Auto Scaling group report it as unhealthy."

Based on this, I expected the ASG to replace the instance when any one Target Group marks it as Unhealthy, as long as ELB health check is enabled.

My question is:

  • Is this the expected behavior?
  • Does the ASG consider an instance Healthy as long as at least one Target Group reports it as Healthy?
  • Or is this a bug, or a limitation not clearly documented?

Thank you for your help.

1 Answer
0

The behavior you're experiencing is not the expected behavior based on the AWS documentation. According to the information provided, when an Auto Scaling group is configured to use Elastic Load Balancing (ELB) health checks, it should consider an instance unhealthy if any of the attached load balancers or target groups report it as unhealthy.

In your scenario, with the Auto Scaling group's health check type set to ELB and one of the two target groups reporting the instance as unhealthy, the Auto Scaling group should mark the instance as unhealthy and initiate the replacement process.

The expected behavior is that if any one of the attached load balancers or target groups reports an instance as unhealthy, the Auto Scaling group should replace the instance, even if others report it as healthy. This is designed to ensure that instances are fully functional across all attached load balancing components.

Given that your Auto Scaling group is not behaving as expected, there are a few things you should check:

  1. Verify that the ELB health check is correctly enabled for your Auto Scaling group.
  2. Ensure that both target groups are properly attached to the Auto Scaling group.
  3. Check the health check settings for both target groups to make sure they are configured correctly.
  4. Review the Auto Scaling group's settings to confirm there are no suspended processes that might be preventing instance replacement.

If after verifying these points, the behavior persists, it may indeed be a bug or an undocumented limitation. In this case, it would be advisable to contact AWS support for further investigation and clarification.

It's important to note that the Auto Scaling group should not consider an instance healthy as long as at least one target group reports it as healthy. The documented behavior clearly states that an instance should be replaced if any attached load balancer or target group reports it as unhealthy.
Sources
Prepare to attach an Elastic Load Balancing load balancer - Amazon EC2 Auto Scaling
Troubleshoot a Classic Load Balancer: Health checks - Elastic Load Balancing

answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.