Issues with traffic from Private VPC > NAT Instance > VPN
Hello there! We are configuring a VPN connection against a financial company using AWS VPN service. Since the company we are connecting to imposed us to use a particular CIDR, we need to NAT the traffic to match our other subnets. Basically we have configured:
- A private VPC in which our K8s cluster lives
- A private VPC for the VPN traffic
- A NAT instance that translates the CIDR traffic between VPCs
- A Transit Gateway that connects VPCs / VPN
- A Site to Site VPN (connection established as expected)
Our networking is:
- VPC K8s: 10.149.0.0/16
- VPC range imposed by the Entity: 10.0.66.0/28
- Entity CIDR: 10.0.64.0/26
From a high level perspective we configured in the NAT instance iptables as follows:
- net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
- iptables -A FORWARD -s 10.149.0.0/16 -d 10.0.64.0/26 -j ACCEPT
- iptables -A FORWARD -s 10.0.64.0/26 -d 10.149.0.0/16 -j ACCEPT
Now that the VPN connection was established, and with this configuration done, the company provided us with a couple of IPs to test ping from within the K8s cluster. In these tests, we are able to reach up to the NAT instance, but the traffic seems to stop there and can't go through. We've already checked SGs, TGW Attachments, NACLs, route tables, VPCs, Source/destination checking...
I know that the information provided might be vague, but we are running out of ideas and any help will be useful.
Let me know if you need further info!
- Newest
- Most votes
- Most comments
Hello.
If you are using a NAT instance, also make sure that "Source/destination checking" is disabled on your EC2 instance.
Even if the security group and iptables settings are correct, if this setting is not disabled, it cannot be used as a NAT instance.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html
Source/destination checking
You can enable or disable source/destination checks, which ensure that the instance is either the source or the destination of any traffic that it receives. Source/destination checks are enabled by default. You must disable source/destination checks if the instance runs services such as network address translation, routing, or firewalls.
Based on your setup, I see a few potential issues that could be preventing traffic from flowing properly through your NAT instance to the VPN.
First, while you've enabled IP forwarding and set up FORWARD rules in iptables, it appears you may be missing the critical NAT (SNAT) rules that would actually translate the source IP addresses. When traffic from your K8s cluster (10.149.0.0/16) is trying to reach the financial company's network (10.0.64.0/26), the source IP needs to be translated to an IP from your allocated range (10.0.66.0/28).
You should add SNAT rules like:
iptables -t nat -A POSTROUTING -s 10.149.0.0/16 -d 10.0.64.0/26 -j SNAT --to-source 10.0.66.x/28
(where 10.0.66.x is a specific IP from your allocated range)
Additionally, check these potential issues:
- Ensure your route tables in both VPCs are correctly configured:
- K8s VPC should route traffic destined for 10.0.64.0/26 to the Transit Gateway
- VPN VPC should route traffic from 10.0.66.0/28 to the Virtual Private Gateway
-
Verify that the Transit Gateway route tables are properly configured to route traffic between the VPCs and to the VPN attachment.
-
Check that the VPN configuration on the AWS side includes the proper static routes for the remote network (10.0.64.0/26).
-
Ensure the remote VPN endpoint has routes that direct return traffic for your NAT'd IP (from 10.0.66.0/28) back through the VPN tunnel.
-
Verify that the security groups for your NAT instance allow both inbound and outbound traffic for the relevant CIDRs.
Since you mentioned traffic reaches the NAT instance but doesn't go further, the most likely issue is either the missing SNAT rules or incorrect routing from the NAT instance to the VPN gateway.
Sources
How Can I Force Traffic from Private Subnets to Appear as a Single NAT private IP for a VPN Site-to-Site Connection? | AWS re:Post
Route Egress Traffic through NAT Gateway to VPN Gateway | AWS re:Post
answered a year ago
Relevant content
asked 2 years ago
Thanks for your response! Yes, we already checked that out.
By the way, have you configured the IP masquerade settings as shown below? "eth0" should be changed to the name of the network interface of your NAT instance.