1 Answer
- Newest
- Most votes
- Most comments
0
If your lambda function is within VPC, then check the subnet security group and make sure it allows outbound traffic to all.
However to communicate with s3 within VPC also, internet is not required and lambda function can directly communicate with S3 via s3 gateway VPC endpoint.
On AWS Console, choose VPC service and then Endpoints. Create a new endpoint, associate it to s3 service. -> Select VPC and Route Table.
Here is the detailed instruction document for same, see section Create a gateway endpoint in this document.
Hope you find this useful.
Abhishek
Relevant content
- Accepted Answerasked a year ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 7 months ago
Thanks for coming back to me - I'm not using a VPC, the lambda function downloads a zip from a website into the temp folder, unpacks the zip into a temp folder which gives me the csv file, the csv file is then uploaded to my bucket. I have maxed out the memory allocation/timeouts. Weirdly enough the file does appear in the s3 bucket but when testing the function gets the error network failure.
When I test the same process but with a smaller zip/csv file, the function works with no errors, so it seems to be some sort of size issue - having said that the csv file when extracted is only 2.4 gbs so still within the max configuration limits of lambda
How are you trying to upload file, direct upload or through streaming(reading chunks of data from s3 file). Would you mind mentioning the API/SDK method here or code snippet. From your description it's clear that it has something to do with size/multipart. This error most likely would come if you'll be using s3.upload_file or s3.put_object, you should rather do it with s3.upload_part or upload_part_copy. See the synopsis and example at here.
Were you able to resolve the issue?