- Newest
- Most votes
- Most comments
Hi,
Based on this page: https://docs.aws.amazon.com/lambda/latest/operatorguide/execution-environment.html
It is impossible to share execution environment across different executions of the same Lambda (for obvious reasons: security, etc.) : see page
When your functions are invoked, the Lambda service runs your code inside an execution
environment. Lambda scrubs the memory before it is assigned to an execution environment.
Execution environments are run on hardware virtualized virtual machines (MicroVMs) which
are dedicated to a single AWS account. Execution environments are never shared across
functions and MicroVMs are never shared across AWS accounts.
But, you can achieve what you want: use AWS Elasticache for Redis. Its recently announced serverless version makes it very cheap.
I use it personally in a current project to share such large amounts of data across different Lambda invocations parts of same Step Function. As you say, it's a huge performance boost.
See https://aws.amazon.com/blogs/aws/amazon-elasticache-serverless-for-redis-and-memcached-now-generally-available/ for details
Best,
Didier
The above mentioned approaches are good. I have a few more recommendations.
Here are a few approaches that can help:
1. Amazon ECS with Fargate:
If you need finer control over instance reuse and in-memory state, consider using Amazon ECS (Elastic Container Service) with Fargate. ECS tasks can maintain a longer-lived state than Lambda, allowing you to keep data in ephemeral storage during the lifecycle of the container across multiple requests. This way, once the data is downloaded for a given DATE, subsequent requests can reuse the temp storage data without the need to fetch it from S3 again. This solution offers more control over how requests are routed and how data is persisted across invocations.
2. Amazon Elastic File System (EFS) with Lambda or ECS:
Lambda now supports integration with Amazon EFS (Elastic File System), which allows you to share a persistent file system across multiple Lambda instances. You could download the data once and store it in EFS, making it accessible to all Lambda instances without needing to fetch it repeatedly from S3. The same approach can be applied in ECS with Fargate, where tasks can be read from EFS to avoid redundant data downloads.
Relevant content
- AWS OFFICIALUpdated 4 years ago
- AWS OFFICIALUpdated 4 years ago

Thank you for the quick response. This is great - I didnt realize Elasticache would work with Lambda. This will help speed things up a bunch
Cheers!
Glad that you like this solution! Thanks for accepting my answer