By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Lambda Instability Sometimes

0

Hi there, my Lambda function sometimes returns task timeout.

My setup: API Gateway Lambda + ECR Image EFS VPC ( to download some items from the internet )

My problem issue: My function works properly after I deploy my image, and make my requests, and I'm using cloudwatch eventbridge to make call requests to my Lambda function every minute.

Sometimes, without a reason, I receive task timeout and it comes and goes after hours of use of the Lambda Function. And when I send my request and receive a task timeout, my function stops at the end and just does not answer with the response, like if the instance didn't start properly and for no reason, can't finish my request ( like if there's some I/O problem ) and I got a "task timeout" error at the end.

Do you have some ideas?

I have already checked my network logs and called my Lambda function directly by the console using the test, and if I'm receiving a "task timeout" error and refresh the Lambda configurations, the error stops.

Circling back to the issue, I have other Lambda functions and it works well with similar configurations. It's not an error that stands, it comes randomly.

I would appreciate any support here,

My best,

Ernani

Topics
ServerlessNetworking & Content DeliveryComputeManagement & Governance
Tags
ServerlessAmazon VPCAWS LambdaAmazon CloudWatch
Language
English
Ernani
asked a month ago462 views
2 Answers
1

I recommend that you add log lines to your function and check in the logs where your function is stuck.

profile pictureAWS
EXPERT
Uri
answered 25 days ago
  • Ernani
    25 days ago

    Hey Uri, thanks for your answer. So I have logs in my function, and the weird thing is that it's like the connection is being dropped after some time executing or something. I will give an example: I have that sequence of logs:

    2024-04-04T10:04:02.333Z checking for cache 2024-04-04T10:04:02.340Z the path already exists - In that case, I don't need to download again and then I get the file from EFS 2024-04-04T10:04:02.466Z You don't set any options 2024-04-04T10:04:02.466Z your data metadata is ... 2024-04-04T10:04:02.613Z init the process 2024-04-04T10:04:02.613Z option set to False 2024-04-04T10:04:02.613Z intermediate results 2024-04-04T10:04:02.619Z execution time: 0.2852690909994635 2024-04-04T10:09:02.485Z 2024-04-04T10:09:02.484Z b8660e79-58f6-4069-b0f1-98f4e4d3f509 Task timed out after 300.14 seconds What is expected here, is the response (it's already done, when we receive the execution time, the difference is 100ms after this message in a normal execution, until you receive your message). But sometimes, out of nowhere, I didn't receive ( it's random )

    I checked my NAT gateway and yesterday my instance was incommunicable for 1 hour ( the active connections count and bytes out graphics go to Zero in that period )

    I have this one period for the day when my instance remains noncommunicable for some reason. Besides that time of the day, it works well.

0
Accepted Answer

I found the error.

There was an issue with my VPC. 2 important things when you attach a VPC to your Lambda: You can't have a public subnet associated. Check if your VPC was created correctly.

Steps: Create a new VPC, using this yaml: https://github.com/awsdocs/aws-lambda-developer-guide/blob/main/templates/vpc-privatepublic.yaml

Or test your Lambda without VPC.

Both tests might work and fix your issue, as it fixed mine.

I hope this helps more people in this community.

Related links: https://repost.aws/knowledge-center/internet-access-lambda-function https://stackoverflow.com/questions/51373609/why-aws-lambda-functions-in-a-vpc-sometimes-timeout-and-sometimes-work-fine

Ernani
answered 17 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content