Skip to content

Why is there a 20 second response time for the first request going to my Application Load balancer? Requests are the first one are less than a second

0

the output of the curl command below curl -kso /dev/null -w "\n===============\n | DNS lookup: %{time_namelookup}\n | Connect: %{time_connect}\n | App connect: %{time_appconnect}\n | Pre-transfer: %{time_pretransfer}\n | Start transfer: %{time_starttransfer}\n | Total: %{time_total}\n | HTTP Code: %{http_code}\n===============\n" https://codereaperdemo.empirestateids.com/codereaper/home

=============== | DNS lookup: 0.008858 | Connect: 21.122347 | App connect: 21.278970 | Pre-transfer: 21.279061 | Start transfer: 21.359980 | Total: 21.360285 | HTTP Code: 200

The EC2 instance has a single Tomcat that the ALB forwards the request to the private EC2 IP. The target group is healthy and has no issues. SSL is terminated at the ALB and sends HTTP on port 8080.

After the first request goes through, subsequent requests are sub-second.

=============== | DNS lookup: 0.145868 | Connect: 0.224109 | App connect: 0.383796 | Pre-transfer: 0.383881 | Start transfer: 0.465919 | Total: 0.466327 | HTTP Code: 200

asked 2 months ago56 views
1 Answer
0

The 20-second delay you're experiencing on the first request is occurring during the connection establishment phase, specifically between the "DNS lookup" (0.008 seconds) and "Connect" (21.122 seconds) stages. This indicates the delay is happening when establishing the TCP connection to your Application Load Balancer, not during DNS resolution.

This pattern - where the first request takes significantly longer but subsequent requests are fast - typically points to a few possible causes:

Connection establishment delay: The first request may be experiencing a delay in establishing the initial connection to the load balancer or from the load balancer to your EC2 target. Since subsequent requests are fast, this suggests connection reuse is working properly after the initial connection is established.

Target initialization: Your Tomcat application on the EC2 instance may need time to initialize or "warm up" on the first request. Even though the target group shows as healthy, the health check may be hitting a lightweight endpoint while your actual application endpoint requires more initialization.

Network path issues: There could be network connectivity issues in the path between the client and the load balancer, or between the load balancer and your EC2 instance. Since you mentioned SSL is terminated at the ALB and forwarded as HTTP on port 8080, check if there are any security group rules, network ACLs, or routing configurations that might be causing delays on initial connections.

To further troubleshoot, I recommend bypassing the load balancer and testing directly against your EC2 instance's private IP address from within the VPC. This will help isolate whether the issue is with the load balancer itself or with your application. Also, check CloudWatch metrics for your load balancer's TargetResponseTime to see if excessive values are being recorded during these first requests.
Sources
Troubleshoot latency on calls or requests to Amazon ECS tasks | AWS re:Post

answered 2 months ago
EXPERT
reviewed 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.