Speeding up a lambda function

0

Hi,

I currently have a lambda function that's using primarily the modules NLTK and LightGBM to do some webscraping and sentiment analysis. However, when I test it, it takes about 7 seconds to run. But when I test it immediately again, it only takes under a second, even with a different website. I'm wondering why this is the case and if it's possible to make all tests run this fast, because it's making my project run pretty slow.

hczhang
asked 9 months ago219 views
2 Answers
1

You should consider provisioned concurrency option for lambda to minimize the latency and cold start. Make sure it has enough memory allocated to it.

Here is the blog post, which is very thorough and have detailed explanation about, how does it work and how it would help you to speed up operation.

To your question, why it takes less time second time, here is the verbiage from above mentioned blog post for your quick reference:

  After the execution completes, the execution environment is frozen. To improve resource management and performance, the Lambda service retains the execution environment for a non-deterministic period of time. During this time, if another request arrives for the same function, the service may reuse the environment. This second request typically finishes more quickly, since the execution environment already exists and it’s not necessary to download the code and run the initialization code. This is called a “warm start”.

For java applications, you can consider snapstart option, here is the blog post for same.

References:

Configuring Provisioned Concurrency

Hope this helps, feel free to comment here if there are additional questions.

Abhishek

profile pictureAWS
EXPERT
answered 9 months ago
profile pictureAWS
EXPERT
reviewed 9 months ago
  • Do you have any questions further?

  • I reserved concurrency for my function which sped it up a lot. Thanks.

0

What you are experiencing is called Cold Start. This happens every time we create a new execution environment to run your function. After the invocation finishes, we keep it around, and if a new invocation happens, we reuse the execution environment, and this is a warm start.

There are different things that can be done to reduce the cold start duration. One of them is to reduce the Lambda package size (may not be possible in your case). You can find more optimization options here.

Note, cold start are most often not an issue. For instance, if your function is invoked asynchronously, cold start time usually doesn't matter. It mainly matters when you have a user making a synchronous API call. In this case, if really needed, you can use provisioned concurrency to initialize execution environments up front. Note that this will incur additional cost.

profile pictureAWS
EXPERT
Uri
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions