Slow lambda responses when bigger load

0

Hi,
Currently, I'm doing load testing using Gatling and I have one issue with my lambdas. I have two lambdas one is written in Java 8 and one is written in Python. I'm using Gatling for my load testing and I have a test where I'm doing one request with 120 concurrent users then I'm ramping them from 120 to 400 users in 1 minute, and then Gatling is doing requests with 400 constants users per second for 2 minutes.
There is a weird behavior in these lambdas because the responses are very high. In the lambdas there is no logic, they are just returning a String. Here are some screenshots of Gatling reports:
Java Report
Python Report
I can add that I did some tests when Lambda is warm-up and there is the same behaviour as well.
I'm using API Gateway to run my lambdas.

Do you have any idea why there is such a big response time? Sometimes I'm receiving an HTTP error that says:

 i.n.h.s.SslHandshakeTimeoutException: handshake timed out after 10000ms 

Here is also my Gatling simulation code:

public class OneEndpointSimulation extends Simulation {

HttpProtocolBuilder httpProtocol = http
        .baseUrl("url") // Here is the root for all relative URLs
        .acceptHeader("text/html,application/xhtml+xml,application/json,application/xml;q=0.9,*/*;q=0.8") // Here are the common headers
        .acceptEncodingHeader("gzip, deflate")
        .acceptLanguageHeader("en-US,en;q=0.5")
        .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0");

ScenarioBuilder scn = scenario("Scenario 1 Workload 2")
        .exec(http("Get all activities")
                .get("/dev")).pause(1);

{
    setUp(scn.injectOpen(
                    atOnceUsers(120),
                    rampUsersPerSec(120).to(400).during(60),
                    constantUsersPerSec(400).during(Duration.ofMinutes(1))
            ).protocols(httpProtocol)
    );
}
}

I also checked logs and turned on the X-ray for API Gateway but there was nothing there. The average latency for these services was 14ms.
What can be the reason for that slow Lambda responses?

1 Answer
0

Hi,

Without your lambda code or lambda logs is hard to say. Are you sure your lambda is taking that long to respond?

You can check the duration using cloudWatch. Try going to cloudWatch > log Insights and runnig this agains your lambda after running your load test.

filter @type = "REPORT"
| stats avg(@duration), max(@duration), min(@duration) by bin(30m)
Fazel
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions