Lambda Timeout When Reading S3 csv

Question

Hi,

I found a very weird issue in my lambda. My lambda code will be hanging on the second time of reading csv ("wr.s3.read_csv(file_path)") and finally time out. If I comment out the 16th line (df_product = wr.s3.read_csv(f"s3://{bucket_name}/processed/product_info/2026/01/06/tb_product.csv")), the csv file reading will success. This is mostly like an issue of awswrangler but I cannot confirm. There isn't any error log returned.

lambda configurations:
Python version: Python 3.13
Architecture: x86_64
Layers: AWSSDKPandas-Python313

```log:
INIT_START Runtime Version: python:3.13.v76	Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:1a6363019b274fc28ffbbec073e5ebf1a872c10c52269844a57c62e74063a49b
START RequestId: 84fe6515-6c72-4171-ad40-0d09bb5183c8 Version: $LATEST
3.14.0
Reading product data
Got 44 product
Reading product package data
END RequestId: 84fe6515-6c72-4171-ad40-0d09bb5183c8
REPORT RequestId: 84fe6515-6c72-4171-ad40-0d09bb5183c8	Duration: 150000.00 ms	Billed Duration: 154340 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	Init Duration: 4339.64 ms	Status: timeout
```

```log comment 16th, 17th line
INIT_START Runtime Version: python:3.13.v76	Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:1a6363019b274fc28ffbbec073e5ebf1a872c10c52269844a57c62e74063a49b
START RequestId: 32c2c100-e2ae-49ee-9441-dc5baaa91b18 Version: $LATEST
3.14.0
Reading product data
Reading product package data
Got 78 product_package
END RequestId: 32c2c100-e2ae-49ee-9441-dc5baaa91b18
REPORT RequestId: 32c2c100-e2ae-49ee-9441-dc5baaa91b18	Duration: 6391.90 ms	Billed Duration: 10739 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	Init Duration: 4346.53 ms	
```

```
import os
import pandas
from datetime import datetime
import awswrangler as wr

import requests

# product document s3 bucket_name
bucket_name = os.environ['bucket_name']

def lambda_handler(event, context):
    # print wr version
    print(wr.__version__)
    # sync_product_and_package()
    print("Reading product data")
    df_product = wr.s3.read_csv(f"s3://{bucket_name}/processed/product_info/2026/01/06/tb_product.csv")
    print(f"Got {len(df_product)} product" )

print("Reading product package data")
    df_product_package = wr.s3.read_csv(f"s3://{bucket_name}/processed/product_package/2026/01/06/tb_product_package.csv")
    print(f"Got {len(df_product_package)} product_package")
    return
```

Riku_Kobayashi · Accepted Answer

Hello.

From the log you provided below, the maximum memory is set to 128MB and the memory used is also 128MB.  
The size of the CSV file you are trying to read is unknown, but it may be that the memory size is too small to read it, resulting in a timeout.  
Try increasing the Lambda memory size.  
https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html  
```
REPORT RequestId: 84fe6515-6c72-4171-ad40-0d09bb5183c8	Duration: 150000.00 ms	Billed Duration: 154340 ms	Memory Size: 128 MB	Max Memory Used: 128 MB	Init Duration: 4339.64 ms	Status: timeout
```

re:Post Agent · Answer

Your Lambda function is timing out when reading the second CSV file from S3. This is happening because of several potential issues:

1. **Memory allocation**: Your Lambda is configured with only 128 MB of memory, which is the minimum allowed. The logs show "Max Memory Used: 128 MB" which indicates you're hitting the memory limit. When working with data processing libraries like pandas and awswrangler, 128 MB is typically insufficient, especially when reading multiple CSV files.

2. **Timeout configuration**: The default Lambda timeout is 3 seconds, but your function is running for the maximum allowed time (150 seconds) before timing out. This suggests your timeout setting is too short for the operation you're performing.

3. **VPC configuration**: If your Lambda function is in a VPC, it might be trying to access S3 without proper network configuration. Lambda functions in a VPC need either an S3 VPC endpoint or a NAT gateway to access S3 efficiently.

4. **File size**: The second CSV file might be larger than the first one, causing the timeout. Large files take longer to download and process.

To resolve this issue:

1. Increase the memory allocation for your Lambda function (at least 1-2 GB for data processing tasks). This will also indirectly improve CPU performance and network bandwidth.

2. Increase the timeout setting to give your function more time to complete.

3. If your Lambda is in a VPC, ensure you have an S3 VPC endpoint configured.

4. Consider implementing streaming reads for large files instead of loading the entire file into memory at once.

5. Check if your error handling in the load_menu_data_from_s3 function is properly logging errors, as you're not seeing any error messages despite the failure.

The fact that commenting out the first CSV read allows the second one to succeed suggests this could be a cumulative memory issue, where the first file's data is still in memory when you try to load the second one.  
**Sources**  
[Why does my Lambda function timeout when accessing S3, even though permissions are correct? | AWS re:Post](https://repost.aws/questions/QU6otiPhsmTcuYwYI80qDr0A/why-does-my-lambda-function-timeout-when-accessing-s3-even-though-permissions-are-correct)  
[Troubleshoot configuration issues in Lambda - AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/troubleshooting-configuration.html)

Lambda Timeout When Reading S3 csv

Add your answer

Relevant content