Failed to upload file to s3

Question

I am currently trying to upload a file to S3 utilizing a Lambda function situated within a VPC, with appropriately configured subnets and security groups. However, I am encountering intermittent 'ETIMEDOUT' errors persistently. The file sizes range between 2MB to 3MB. Below, I have provided the code snippet I am employing for the upload operation.
let fileStream = fs.createReadStream(fileToUpload);
let data = await  s3
    .upload({
      Bucket: bucket,
      Key: fileKey,
      Body: fileStream,
    }).promise();

Riku_Kobayashi · Answer

Hello.

Have you set up an S3 VPC endpoint?  
Timeout errors may occur because there is no access route to S3.  
Therefore, please check whether a route to the NAT Gateway or a route to the S3 gateway VPC endpoint is set in the route table of the subnet linked to Lambda.  
https://repost.aws/knowledge-center/internet-access-lambda-function  
https://docs.aws.amazon.com/lambda/latest/dg/foundation-networking.html#foundation-nw-connecting

Also, try increasing the Lambda timeout settings to improve it.  
https://docs.aws.amazon.com/lambda/latest/dg/configuration-timeout.html

Leo K · Answer

The exception thrown with the ETIMEDOUT code should contain a more exact statement about what failed. Is it a connection error or something else?

I expect you have some form of connectivity in place between your Lambda function and S3, considering you described the issue as intermittent, but there could still be a partial issue, such as some of the subnets the Lambda function is attached to not having a route to S3.

If your S3 bucket is in the same region as the Lambda, the best and least expensive way to connect is by creating a VPC gateway endpoint for S3 in  your VPC and telling it to add the required routes towards S3 in all the route tables in the VPC, or at least the ones that will be connecting to S3. Gateway endpoints for S3 don't charge for network traffic, unlike NAT gateways.

You should also check if the DNS option set associated with your VPC is set to use "AmazonProvidedDNS" or 169.254.169.253 or something else for DNS resolution. Connectivity or timeout issues with the Lambda querying the DNS names for S3 could also cause this symptom.

Didier_Durand · Answer

Hi,

Go to CloudTrail and check for the API calls made by your Lambda function to see if any fails (authorization credentials, etc.)

That will probably help you find the root cause of your problem.

Best,

Didier

Failed to upload file to s3

Relevant content