lambda first call too slow

0

I am running a C++-based lambda that attatched to EFS and read binary files. However, the first API call takes about 20~30 seconds. The next call is performed within 10 seconds. I set "provisioned concurrency" because I thought it was just a coldstart issue, but the first call is still slow.

Could it have something to do with the file cache? And is there any solution to this?

Soo
posta un mese fa180 visualizzazioni
3 Risposte
3

Provisioned Concurrency helps mitigate cold starts by keeping a set of initialized execution environments ready to handle incoming requests. However, in your case, the initial delay seems to be related to the time required to mount the Elastic File System (EFS) and populate the file cache, rather than a cold start issue.

When the Lambda function needs to read files from the EFS for the first time or after a prolonged period of inactivity, it has to establish a connection to the EFS and load the required file data into the cache. This process can be time-consuming, especially if the files are large or if there is higher latency in accessing the EFS.

Provisioned Concurrency alone may not be sufficient to address this issue because it doesn't affect the time required for mounting the EFS and populating the file cache during the initial invocation.

The subsequent invocations are faster (around 10 seconds) because the EFS is already mounted, and the necessary files are cached in memory, allowing for quicker access.

So, while Provisioned Concurrency can help reduce cold starts, it doesn't directly address the delay caused by the initial EFS mounting and file caching process, which seems to be the root cause of the slow first API call in your case.

profile picture
ESPERTO
con risposta un mese fa
profile picture
ESPERTO
verificato un mese fa
profile picture
ESPERTO
verificato un mese fa
1
Risposta accettata

Try reading a file in the initialization code. This way you will pre-warm the EFS access during function deployment and not in the first execution.

profile pictureAWS
ESPERTO
Uri
con risposta un mese fa
profile picture
ESPERTO
verificato un mese fa
  • Thanks for the advice. I opened the file outside the handler function, read the file a little, and did not close it. The first function call time has been reduced about 3 seconds. Maybe if I read more, I could save more time.

  • I inserted the code to copy file from EFS to temporary storage in the init code area. So the handler function read the file from the emphemeral(/tmp/) storage and was able to achieve very fast speed.

    There is a lot of work to be done in the static code area, such as transferring and decompressing files. I set provisioned concurrency, so I was able to avoid init in the actual API call.

    Thank you again for giving me inspiration and solutions to my problems.

1

It could be that your API Gateway(?) call is not using the provisioned Lambda. If you have logging enable, you can check from CloudWatch log group for INIT_START

Do ensure that your API call and provisioned are referring to the same version.

AWS
ESPERTO
Mike_L
con risposta un mese fa
  • Yes, I am using an API gateway, and I set it to use the versioned lambda before deploying. But it's still slow, the loading process from EFS seems to take a long time. Thank you for answer.

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande