Benefits to S3 cross-region access with VPC peered interface endpoints vs. public internet using NAT gateways?

Question

My team is looking to setup EMR clusters in private VPCs in all regions while having our main storage as S3 buckets in us-east-1. We will need cross-region access to S3 and have been looking at different ways of accomplishing it. We have considered two approaches:

1. Setting up isolated VPCs with no internet access, one in us-east-1 for the S3 bucket access and one in every region to launch our EMR clusters in. We will pair each of the VPCs with the one in us-east-1 and then setup an interface endpoint in the us-east-1 VPC to allow S3 access through the interface endpoint with VPC peering. This utilizes AWS PrivateLink.

2. Setting up a private VPC with internet gateway and NAT gateways in public subnets while launching EMR clusters in the private subnets. We will access S3 across regions through public internet.

For both solutions, we will utilize gateway endpoints when the compute and storage is in the same region as we found this should yield the same benefits as interface endpoints but with no additional cost.

Through my research, I have found that AWS PrivateLink is more secure due to no public internet usage and has a significant latency advantage of up to 70% according to this experiment: https://blogs.vmware.com/security/2020/03/performance-testing-justifying-cost-and-performance-improvements-part-2.html

I am wondering if we will still see this latency benefit if we are using VPC peering or if it would be better to go with the internet route.

Accepted Answer

Bear in mind that using public IP addressing does not necessarily mean "public internet". Indeed, looking at the [VPC FAQ](https://aws.amazon.com/vpc/faqs/) we state `When using public IP addresses, all communication between instances and services hosted in AWS use AWS's private network.` In reality, because S3 uses TLS the security benefit here comes down to the endpoint policies allowing you to restrict access to specific S3 buckets rather than anything else.

And because the traffic between regions goes over the same backbone network whether you are using public IP addresses or private IP addresses (via VPC or Transit Gateway peering) the latency difference will be negligible.

S3 Gateway Endpoints are zero cost but can only be accessed from within the VPC that they are created in. S3 PrivateLink endpoints can be accessed from other peered VPCs but they do come with a cost to do that.

In summary: Choose the architecture which is lowest cost and meets your requirements.

Answer

I would go with the VPC peering and interface endpoint.

Have you considered using S3 replication to each region? Then have a Gateway endpoint on each VPC.

Benefits to S3 cross-region access with VPC peered interface endpoints vs. public internet using NAT gateways?

Relevant content