ALB path based routing 500 Errors

Question

We have onboarded a new microservice to take on traffic to a few endpoints and are routing traffic to this microservice through the original service's ALB and path based routing. Overall things look good, requests are being routed through the ALB to the correct service depending on which endpoint is being hit.

However one interesting thing we have begun to notice is there is a small but steady stream of 500 errors on the ALB. It is much less than even 1% of total requests are seeing 500 errors. When checking application logs we do not see anything that correlates this and have not found much in the ALB logs either.

These 500 errors were not present on the ALB when traffic was all being routed to a single service so we are wondering if the path based routing is maybe causing these issues or is there a potential issue in the new microservice. Any insight would be appreciated.

Answer

Hi,

While it's difficult to pinpoint the exact cause without more information, here are some suggestions for troubleshooting and narrowing down the culprit:

* Consider conducting load testing on your microservice to simulate production traffic and identify any performance bottlenecks or scalability issues. Pay attention to response times and error rates under different levels of load. https://aws.amazon.com/blogs/architecture/ensure-optimal-application-performance-with-distributed-load-testing-on-aws/
* I assume the health checks or monitoring for your microservice is taken care of. 
* Ensure that your microservice is properly logging errors and exceptions, including those that result in 500 errors.
* Use tools like curl or Postman to manually send requests to the microservice endpoints that are being routed through the ALB. This can help identify any specific conditions or inputs that trigger the 500 errors.
* Check how the ALB is handling error responses from the microservice. Ensure that it's returning appropriate HTTP status codes and error messages to clients
* Even though you mentioned you didn't find anything in the application logs, it's still worth investigating further. Look for any errors or exceptions that might be occurring in the microservice logs. Pay attention to any patterns or specific endpoints that seem to trigger the 500 errors.

LMK if anyone of these helps.

ALB path based routing 500 Errors

Relevant content