- Newest
- Most votes
- Most comments
What you're describing isn't possible.
You're saying you don't want the ALBs in the non-overlapping CIDRs prescribed in some of the blog posts. That would leave you with an instance in VPC A, having a 10.0.x.y IP address, connecting to a target instance in VPC B, also having a 10.0.a.b IP address. That packet will never leave VPC A -- because VPC A considers the packet's destination IP 10.0.a.b as local to VPC A.
NAT gateways, both private and public, only hide the source IP of a connection behind one of the IPs of the NAT gateway. Your problem is with the destination IPs overlapping, which cause the traffic between VPCs to be considered as local to the source VPC. Therefore, the traffic will never even get sent to a NAT gateway, and even if it did, it would still get delivered to the original source VPC, because the destination IP would be unchanged.
You would have to have an ALB, NLB, custom middlebox, or something else with a non-overlapping IP to serve as the destination IP address of the connection coming from VPC A that distinguishes it from the IP addresses local to VPC A. That's the purpose of the ALBs in some of the workarounds described in the blog posts. If you had an ALB in VPC B in an IP range 10.221.64.0/22, for example, it wouldn't overlap. You still wouldn't be able to peer the VPCs, but you could route the non-overlapping range through a transit gateway (TGW). You would also place the private NAT gateways in the non-overlapping CIDRs.
When the instance in VPC A with the source IP 10.0.x.y would connect to the ALB in VPC B with the 10.221.64.0/22 IP, the destination would be known not to be local to VPC A. That's what would allow it to be routed to the private NAT gateway in VPC A. The private NAT gateway would have a non-overlapping IP in VPC A, such as 10.221.68.0/22. After processing on the private NAT gateway, the packet would have a source IP under 10.221.68.0/22, belonging to VPC A, and a destination IP under 10.221.64.0/22 that belongs to VPC B.
Relevant content
AWS OFFICIALUpdated 3 years ago
Thanks for the detailed explanation! It definitely clarifies the issue with overlapping destination IPs and why a Private NAT Gateway wouldn't work on its own.
Based on your explanation, using a Private NAT Gateway in conjunction with an Application Load Balancer (ALB) with a non-overlapping CIDR seems like a viable approach. The blog post provides a detailed walkthrough of this method. Since I need to make the subnet private anyway, another option would be to create a new VPC with a non-overlapping CIDR. This would allow for simpler inter-account VPC peering without needing additional configurations like the NAT Gateway and ALB. I will evaluate it further.
Yes, but to be clear, let's say that you'd create a new VPC, "VPC C", with the non-overlapping CIDR 10.221.96.0/22, for the sole purpose of hosting the interconnection components, like ALBs and NLBs. You would be able to peer VPC C with either VPC A's 10.0.0.0/16 or VPC B's 10.0.0.0/16, but you won't be able to peer or otherwise connect VPC C with both VPC A and VPC B, because 10.0.0.0/16 could meaningfully only be routed towards one or the other. If you added VPC D with 10.221.128.0/22, you could peer VPC A+C, B+D, and C+D to create a U-shaped construct, though.
you answer would be the least cost effective but then Can we use an EC2 instance with an IP in the secondary CIDR block of VPC A to connect directly with another EC2 instance in the secondary CIDR block of VPC B?
Yes, to the extent that when the secondary CIDR blocks of VPCs A and B do not overlap, you can establish connectivity between them through a Transit Gateway. However, you cannot establish a VPC peering connection between VPC A and B despite some of the CIDRs not overlapping, because VPC peering connections cannot be established if any IPs overlap between the VPCs. Operationally (not considering the transformation cost), the most cost-effective option is to move resources from at least one VPC to a new VPC with non-overlapping IPs and establish a regular VPC peering connection.
Note that you still have the alternate option of establishing PrivateLink connectivity, also explained in some of the earlier blog posts. For example, for a destination X in VPC B, you'd set up an NLB in VPC B with X as its target. You'd "publish" the NLB as a VPC endpoint service and create a VPC endpoint for it in VPC A. From a workload in VPC A needing access, instead of connecting straight to X, you'd connect to the local VPC endpoint in VPC A instead. There are additional costs for the NLB, VPC endpoint, and traffic passing through them, however, and VPC endpoints are limited to TCP only.