- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
Have you created an Interface VPC endpoint to S3 in your VPC. If you haven't the calls to S3 from the lambda function will be made over the Internet. If you have then the calls will be made over the Amazon network and will therefore be of much lower latency - https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html
Hello, Indranil is correct in his comment, you should really set up a private gateway to s3 (https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html). It does not cost you anything, and it will increase network throughput while offloading your NAT.
However, the architecture you have should allow for faster traffic than you get, so my best guess would be the client transfer config, this is the python example, but every SDK (and the CLI) has its own version. https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3.html
Hope it helps!
Thanks but the issue isn't the S3 upload for the small file, but rather the download over http from a third party seem to choke at 1mb/s. I'm streaming multipart uploads on S3 so that's taken care of.
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata un anno fa
Thanks that's helpful, but my question isn't actually about the S3 upload, it's just odd to me that it's slow. The actual issue is that I'm fetching a file from a third party over HTTP, and only seeing around 1mb/s down even though my home network is doing 7mb/s down.
How much memory is allocated to your lambda function? You can try increasing the memory to see if you get better performance.
At 1GB I get 1mb/s. At 2GB I get 2mb/s. At max 10GB I get 2mb/s. Also I've confirmed there is a VPC Endpoint Gateway setup for S3 already, and it's mapped to the private subnet and the lambda can reach the gateway endpoint according to the reachability analyser.