AWS ELB lots of 5xx when facing high peak

Question

I am experiencing a high number of ELB 5xx errors where it did not forward the traffic to my ECS targets, during a high peak where access went up from 10k to 60k per minute.

After research we are quite confident that it is not anything in here: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-troubleshooting.html. And also the ECS service itself is running well.  Anyone has any idea what could be the bottleneck for ELB?

Answer

You may be experiencing a ramp up so fast the ALB is not scaling fast enough. Look at the following [metrics](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-cloudwatch-metrics.html):

ActiveConnectionCount  
NewConnectionCount  
RejectedConnectionCount

If you are using a Classic Load Balancer, Look at the following [metrics](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-cloudwatch-metrics.html):

RequestCount  
SpilloverCount  
SurgeQueueLength

If you see that you are not scaling fast enough, you can work with support to "pre-warm" the load balancers by filing a support ticket.

AWS ELB lots of 5xx when facing high peak

Relevanter Inhalt