Lambda Stability issue

This sounds like a code error. When you invoke a function for the first time, we initialize an execution environment. When you invoke it again, we reuse the previous execution environment. When you make changes, we create a new one.

It is possible that you have some memory leak, or some other code mistake, that causes the code to get into a loop when invoking several times. I recommend that you add some debug messages and try it again to find out where it is egtting stuck in your code.

How are you triggering the lambda? Have you investigated the cloudwatch logs? What are the metrics showing?

It sounds like you are timing out as its taking too long to run. This could be many reasons such as a resource you are referencing is timing out itself. Your function may be performing too much and is not having time to finish.

You cant assign a Public IP address to a Lambda function. Security groups are for outbound access only with Lambda. Theres no inbound traffic for lambda. What do your subnets and route tables look like? Is your ENI on a Public or Private Subnet?

I don't know if your issue is the same as I got in my environment. I was thinking about opening a new post, and then I saw you that looks like the same issue I'm figuring now.

I have now an environment with a lambda, that is based on an ECR image. So it's not the only one, I have other environments with this setup.

That said: My environment consists of:

1. API Gateway with LAMBDA integration
2. EFS
3. VPC ( it's working properly )
4. ECR Image

What is my issue: When I made the first requests while Lambda was warmed up, it worked very well. But after some while, we started to have some instability during the requests, with task timeout.

What I tested: I changed the Lambda timeout to check if it's taking longer than necessary to execute ( does not make sense in my situation, because the process reaches the end ) and it doesn't finish and answers back with the generated response. I think it should be the connection between Lambda and API Gateway, that could maybe not start the ENI properly. So then I tested the function call using the console, and I configured a test call. I continued receiving task timeout, the same issue. But it only happens in two situations: If the Lambda function is idle Sometimes, after a long idle time, when I try to make requests, the Lambda function gets something like 1 hour to stop the task timeout issue.

Now, I'm trying to let the instance warm, and I think it could fix my issue.

I tried debugging my code, looking after the EFS throughput, and increasing my instance size, and what I noticed is, that if the instance is already available, it answers sometimes in ms, depending on the request. And the problem continued even though I used provisioned instances as well.

So I don't know if it's the same issue you are figuring, but I have had 2 conclusions, correct me if I'm wrong: During the day, if your instance is idle, AWS manages the resources ( Lambda is serverless, so you need to redistribute what's not being used, Am I right? ) and that's when I have 1 hour of unavailability. So if in my situation I need more availability, I need to warm up my lambda or use another AWS service, like ECS.

Did this answer your question, or help figuring out something new?

What are your AWS experts' thoughts about this?

Warm regards

Please refer to my post, where I fixed a similar issue:

https://repost.aws/questions/QU75AxAy1eSD29YF2rUTfl2g/lambda-instability-sometimes