Reaching max memory size after several re-invocation of the same lambda

Question

I have a lambda performing some reads and writes to a database.
Several instances of my lambda are running in parallel.

After processing one item, a new invocation is launched for each instance as described in the documentation (https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html)
Why does the memory used increase after each invocation until it reaches the limit, causing a 'RunTimeExit.Error'?

[EDIT - Solved]

It turns that the problem was coming from the pandas SeriesGroupBy.rolling() method operating on an entire grouped DataFrame instead of only some specific columns.

Didier Durand · Answer

Hi,

This troubleshooting guide will provide you some paths to explore to find the root  cause of your issue: https://repost.aws/knowledge-center/lambda-runtime-error

Best,

Didier

Srini V · Answer

Apart from answer mentioned above by Didier, the increasing memory usage pattern in Lambda functions can be attributed to garbage collection (GC) behavior and memory allocation. When Lambda functions have lower memory settings, GC runs more frequently which can impact performance . For Lambda functions experiencing memory issues, it's important to first confirm if this is a single occurrence or a recurring pattern, and then use CloudWatch insights to analyze memory usage patterns over time by checking average and maximum memory utilization . Monitor (profile) and find the root cause. See if this post helps.

https://repost.aws/knowledge-center/lambda-function-memory-usage-monitoring

Reaching max memory size after several re-invocation of the same lambda

Add your answer

Relevant content