AWS Lambda function in VPC with S3 gateway endpoint randomly times out



For the testing purposes I have created a simple lambda function with the following configuration:

  • runtime: Python 3.11
  • memory: 128 MB
  • timeout: 20s
  • subnets: us-east-1b, us-east-1a (public)
  • security group: inbound/outbound allow all trafic

S3 gateway configuration:

  • route table associated with subnets in us-east-1b, us-east-1a (public)
  • added route table route for s3 endpoint destination

The function code is:

import json
import boto3

def lambda_handler(event, context):
    print("CREATE CLIENT")
    s3 = boto3.client("s3")
    print("START REQUEST")
    resp = s3.list_objects(Bucket="xbrl-dev20230712203329631801")

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')

The logs from last several executions:

Enter image description here

The logs show that the function timed out 5/9 times. While it worked 4/9 times (I got the list_bucket response).

Execution results for the timed out invocation:

Test Event Name

  "errorMessage": "2023-09-23T09:43:34.149Z be338960-e7a1-48af-b438-7ea6646f0eef Task timed out after 20.02 seconds"

Function Logs
START RequestId: be338960-e7a1-48af-b438-7ea6646f0eef Version: $LATEST
2023-09-23T09:43:34.149Z be338960-e7a1-48af-b438-7ea6646f0eef Task timed out after 20.02 seconds

END RequestId: be338960-e7a1-48af-b438-7ea6646f0eef
REPORT RequestId: be338960-e7a1-48af-b438-7ea6646f0eef	Duration: 20023.08 ms	Billed Duration: 20000 ms	Memory Size: 128 MB	Max Memory Used: 79 MB	Init Duration: 314.01 ms

I would appreciate any hints regarding why this lambda function is randomly not working. I understand that I could get rid off all that issues by not deploying in a VPC, but I want to connect to a database in VPC from my lambda.

-----------------------EDIT ---------------------

I have increased the function timeout to 5 minutes and now all the timed out requests work but they take around 60 seconds each. The size of the response is around 5kb, That performance is unacceptable.

-------------------- SOLUTION ----------------------

Deploy lambda function in a private subnet associated with a route table without a route to internet gateway (just use the S3 VPC endpoint route).

asked 10 months ago429 views
1 Answer

Make sure you use PrivateLink and not going via the internet

profile picture
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions