- Newest
- Most votes
- Most comments
The difference between the expected MSS of 8460 bytes (based on MTU of 8500) and the observed MSS of 8365 bytes is likely due to additional protocol overhead and encapsulation that occurs within AWS networking services.
Transit Gateway supports an MTU of 8500 bytes for traffic between VPCs, AWS Direct Connect, Transit Gateway Connect, and peering attachments. Similarly, NAT gateways support traffic with a maximum MTU of 8500 bytes. Both services enforce Maximum Segment Size (MSS) clamping for all packets.
When traffic passes through these services, there may be additional encapsulation required for routing and security purposes that consumes some of the available packet size. This encapsulation overhead reduces the effective payload size available for your data, resulting in a lower MSS than the theoretical maximum.
It's important to note that while AWS publishes the MTU as 8500, this represents the maximum packet size that can traverse the service, including all headers and encapsulation. The usable payload space (reflected in the MSS) will necessarily be smaller to accommodate these headers.
For context, when migrating from VPC peering (which supports jumbo frames with MTU of 9001 in the same region) to Transit Gateway, you previously had to manually configure the MTU to less than 8500 on EC2 instances to prevent traffic from being silently dropped. Recent enhancements to Transit Gateway now support Path MTU Discovery (PMTUD), which helps manage these MTU differences automatically by sending appropriate ICMP messages when there's an MTU mismatch.
If you're experiencing specific performance issues related to the MSS value, you might want to consider adjusting your application to work optimally with the observed MSS rather than the theoretical maximum.
Sources
Amazon VPC Transit Gateways Quotas - Amazon VPC
Performance and metrics enhancements for AWS Transit Gateway and AWS Cloud WAN | Networking & Content Delivery
NAT gateway basics - Amazon Virtual Private Cloud
When traffic passes through these services, there may be additional encapsulation required for routing and security purposes that consumes some of the available packet size. This encapsulation overhead reduces the effective payload size available for your data, resulting in a lower MSS than the theoretical maximum.
That is why the MTU is not 9001 but much lower (8500)! Still, it does not explain why the MTU of 8500 is advertised when the actual (IP) MTU seems to be 8405.
I tested again:
- TGW will do MSS clamping to 8460 as expected. Even with two TGWs in the path, it stays at 8460.
- A private endpoint -> private link service -> NLB will do MSS clamping to 8365! According to the documentation it should do 8460 instead.
- In a centralized inspection setup with a TGW + appliance mode, then an Inspecion VPC with GWLB endpoints -> GWLB > NVA appliance I see this behavior:
-
- SYN arrives at FW NVA with MSS clamped to 8365 (like I tested, its probably the private endpoint of GWLB type doing that)
-
- SYN packet goes back from FW with MSS still at 8365.
-
- SYN packet arrives at destination with MSS now at 8341 (something clamped it down by 24 bytes again!?)
Your observation is correct, it seems the GWLB endpoint is clamping MSS at a lower value (8365 bytes). We will work on getting this documented accurately.
Relevant content
- asked 6 years ago
