AWS Site-to-Site VPN ping working, TCP not (EC2 networking details)
I want to establish a site-to-site IPsec VPN connection between an AWS EKS-Kubernetes-Cluster and a server from a different provider using AWS Site-to-Site VPN. Pings get through the VPN, but TCP traffic does not.
The server on the other end runs Ubuntu 20.04 and uses libreswan. The configuration file from AWS for the VPN for openswan has been altered in two ways (that I think should not matter):
auth=esphas been commented out as libreswan would not start otherwise (libreswan 3.29)- The VPN has been configured to use VTI.
When sending a HTTP request from the AWS site: tcpdump on the libreswan-site shows SYN arriving and SYN-ACK being sent back while tcpdump on the EC2-instance (and in a pod as well) only registers SYN.
All incoming traffic has been allowed in security groups and ACLs etc.
From my understanding of the flow logs, the problem occurs within an EC2 instance:
- The pod runs on the instance with its own ip address.
- Network traffic of the pod passes through the network interfaces of the underlying EC2 instance.
- Pings are processed correctly, but other traffic is not forwarded from the pod to the network interfaces of the EC2 instance.
Is my understanding correct? How does networking work inside an EC2 instance with EKS pods?
Hello,
Your understanding is correct, it is explained here
By default, the source IPv4 address of each pod that communicates with resources outside of the VPC is translated through network address translation (NAT) to the primary IP address of the primary network interface attached to the node. You can change this behavior to instead have a NAT device in a private subnet translate each pod's IPv4 address to the NAT device's IPv4 address.
The fact that Ping (echo and echo-reply) is working I think the VPN is setup properly. You may want to look at this link which explains SNAT in detail and inbound/outbound traffic to and from PODs.
Relevant questions
Is it possible to set up a dynamic routing connection to AWS through a site-to-site VPN via a vendor?
Accepted Answerasked 2 years agoAWS VPN Client - how does it open a browser for SSO ?
asked 4 months agoAWS Site-to-Site VPN ping working, TCP not (EC2 networking details)
asked a month agoWanted VPN tunnel between elastic ip and on prem static IP?
asked 2 months agoAdvice on creating VPC for EC2 to use IPSec connection
asked 4 months agoHow to establish connection between AWS Fargate task to an external SFTP server?
asked 2 months agoAWS Site-to-Site VPN ping working, TCP not
asked 19 days agoConnection to external VPN from Windows Server 2016
asked 5 months agoSetup an AWS account
asked 6 months agoWorking around AWS VPN MTU limits
Accepted Answerasked 2 years ago
Thank you, that brought me closer to a solution. I have set up a (public) NAT gateway in a public subnet and use it for NAT for pods in a private subnet. However the behavior is the same: Ping works and TCP does not.
tcpdumpandtracerouteshow that the traffic is going through the NAT gateway and the VPN tunnel. Somewhere along the way the answer gets lost/discarded. I assume now the issue is no more EKS related as the NAT works fine for pings and non-VPN connections.