Skip to content

Long delay before Lambda receives data after InvokeCommand is sent from the frontend

0

For about 5% of all requests we see that after Lambda InvokeCommand is sent from our frontend Using aws-sdk-js-v3 on React) it takes several seconds, sometimes even 10 or more, before Lambda actually receives data (i.e. START log entry is seen in the Cloudwatch logs). The issue is not related to Lambda cold starts as this happens on warm Lambdas (where previous request has finished couple seconds before). Any recommendations how this can be prevented or solved?

asked a year ago140 views

1 Answer
0

This delay between sending an InvokeCommand and Lambda actually receiving the request (as evidenced by the START log entry) could be caused by several factors:

  1. Network latency: There might be network delays between your frontend and AWS services, especially if your users are geographically distant from the AWS region you're using.

  2. SDK initialization: When making AWS SDK calls, there can be delays related to SDK initialization, particularly for the first few operations. This includes loading classes and setting up secure connections.

  3. Request throttling or service limits: If you're sending many requests in a short period, you might be hitting service limits or throttling, causing some requests to be delayed.

  4. Lambda service internal processing: There can be internal processing delays within the Lambda service itself before your function is invoked.

To address this issue, consider these approaches:

  1. Implement retry logic with exponential backoff: This can help handle transient issues more gracefully.

  2. Monitor and analyze patterns: Look for patterns in when these delays occur. Are they happening at specific times or under certain load conditions?

  3. Increase memory allocation: Sometimes allocating more memory to your Lambda function can improve overall performance, including reducing latency.

  4. Use Provisioned Concurrency: This keeps execution environments warm and ready to respond immediately to invocations, which can help with consistent performance.

  5. Consider AWS Lambda SnapStart: If you're using Java, Python, or .NET, SnapStart can improve startup performance by taking a snapshot of the initialized execution environment.

  6. Optimize your frontend code: Ensure your React application is efficiently handling the AWS SDK calls and not introducing additional delays.

  7. Check CloudWatch metrics: Monitor the Lambda invocation metrics to identify any patterns or anomalies that might explain the intermittent delays.

Since this issue occurs only for about 5% of requests and on warm Lambdas, it suggests an intermittent issue rather than a systematic problem with cold starts. This makes it particularly important to gather more data about when and under what conditions these delays occur.
Sources
Lambda delay before any request is made | AWS re:Post
Lambda has performance issue | AWS re:Post
Better handling on AWS Lambda on cold starts | AWS re:Post

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.