lambda first call too slow

0

I am running a C++-based lambda that attatched to EFS and read binary files. However, the first API call takes about 20~30 seconds. The next call is performed within 10 seconds. I set "provisioned concurrency" because I thought it was just a coldstart issue, but the first call is still slow.

Could it have something to do with the file cache? And is there any solution to this?

Soo
asked a month ago153 views
3 Answers
3

Provisioned Concurrency helps mitigate cold starts by keeping a set of initialized execution environments ready to handle incoming requests. However, in your case, the initial delay seems to be related to the time required to mount the Elastic File System (EFS) and populate the file cache, rather than a cold start issue.

When the Lambda function needs to read files from the EFS for the first time or after a prolonged period of inactivity, it has to establish a connection to the EFS and load the required file data into the cache. This process can be time-consuming, especially if the files are large or if there is higher latency in accessing the EFS.

Provisioned Concurrency alone may not be sufficient to address this issue because it doesn't affect the time required for mounting the EFS and populating the file cache during the initial invocation.

The subsequent invocations are faster (around 10 seconds) because the EFS is already mounted, and the necessary files are cached in memory, allowing for quicker access.

So, while Provisioned Concurrency can help reduce cold starts, it doesn't directly address the delay caused by the initial EFS mounting and file caching process, which seems to be the root cause of the slow first API call in your case.

profile picture
EXPERT
answered a month ago
profile picture
EXPERT
reviewed a month ago
profile picture
EXPERT
reviewed a month ago
1
Accepted Answer

Try reading a file in the initialization code. This way you will pre-warm the EFS access during function deployment and not in the first execution.

profile pictureAWS
EXPERT
Uri
answered a month ago
profile picture
EXPERT
reviewed a month ago
  • Thanks for the advice. I opened the file outside the handler function, read the file a little, and did not close it. The first function call time has been reduced about 3 seconds. Maybe if I read more, I could save more time.

  • I inserted the code to copy file from EFS to temporary storage in the init code area. So the handler function read the file from the emphemeral(/tmp/) storage and was able to achieve very fast speed.

    There is a lot of work to be done in the static code area, such as transferring and decompressing files. I set provisioned concurrency, so I was able to avoid init in the actual API call.

    Thank you again for giving me inspiration and solutions to my problems.

1

It could be that your API Gateway(?) call is not using the provisioned Lambda. If you have logging enable, you can check from CloudWatch log group for INIT_START

Do ensure that your API call and provisioned are referring to the same version.

AWS
EXPERT
Mike_L
answered a month ago
  • Yes, I am using an API gateway, and I set it to use the versioned lambda before deploying. But it's still slow, the loading process from EFS seems to take a long time. Thank you for answer.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions