PMTUD on Transit Gateway

Question

Dear Team - I was going through   **https://aws.amazon.com/blogs/networking-and-content-delivery/performance-and-metrics-enhancements-for-aws-transit-gateway-and-aws-cloud-wan/**  blog.

As per above ,  **When there is a difference in MTU sizes in the network between two hosts, PMTUD enables the network device (Transit Gateway/CNE) in the path to respond to the originating host with an Internet Control Message Protocol (ICMP) message. This ICMP message instructs the originating host to use the lowest MTU size along the network path and resend the request.**

i want to understand how above feature help on below two use cases mentioned in same blog.

Does it mean, with PMTUD, now packet with higher than 8500 MTU allowed through TGW ? if yes, how ? Will the TGW not send the ICMP message back to sender to adjust the lower MTU ?

The default Amazon Linux Amazon Machine Image (AMI) uses an MTU of 9001 (jumbo frames). This feature allows users to continue using jumbo frames MTU (9001) on Amazon Elastic Compute Cloud (Amazon EC2) instances while sending traffic through Transit Gateway/AWS Cloud WAN, without having to manually configure Linux/Windows instances to use an MTU lower than the Transit Gateway/CNE supported value of 8500.

Jumbo frames are supported for VPC peering connections within the same AWS Region. Previously when users migrated from VPC peering to Transit Gateway, traffic would be silently dropped, necessitating that users manually configure the MTU to less than 8500 on potentially hundreds or thousands of EC2 instances across VPCs before migration. This manual configuration is no longer needed.

Tushar Jagdale · Accepted Answer

> i think,  if host can adjust automatically MTU once receive ICMP message from TGW,  then it should have clarify and call out in original blog post ?

If you look at the diagram in the blog (Figure1), see the flow # 2 which explains this behavior.

re:Post Agent · Answer

Path MTU Discovery (PMTUD) on Transit Gateway helps manage MTU mismatches between connected networks by providing feedback to sending hosts rather than silently dropping packets.

To clarify how this works:

When a packet with an MTU larger than 8500 bytes (Transit Gateway's maximum) arrives at the Transit Gateway, the packet is not allowed through as-is. Instead, with PMTUD support, the Transit Gateway now sends back an ICMP message - either "Fragmentation Needed" for IPv4 or "Packet Too Big" for IPv6. This notifies the sending host that it needs to adjust its packet size.

For your specific use cases:

1. EC2 instances using Amazon Linux AMI with 9001 MTU (jumbo frames): Previously, when these instances sent packets larger than 8500 bytes through Transit Gateway, the packets would be silently dropped, causing connectivity issues. Now, with PMTUD, the Transit Gateway sends back an ICMP message instructing the host to reduce the packet size to 8500 bytes or less. The host's operating system can then automatically adjust future packets to the appropriate size without manual configuration.

2. VPC peering migration scenario: When migrating from VPC peering (which supports 9001 MTU within the same region) to Transit Gateway (which supports 8500 MTU), previously administrators had to manually reconfigure MTU settings on all EC2 instances before migration to avoid packet loss. With PMTUD support, this manual reconfiguration is no longer necessary because the Transit Gateway will notify hosts when packets are too large, allowing them to automatically adjust.

The key benefit is that PMTUD eliminates silent packet drops and allows hosts to automatically adapt to the Transit Gateway's MTU limitations without manual intervention, making network configurations more resilient and migrations simpler.  
**Sources**  
[Performance and metrics enhancements for AWS Transit Gateway and AWS Cloud WAN | Networking & Content Delivery](https://aws.amazon.com/blogs/networking-and-content-delivery/performance-and-metrics-enhancements-for-aws-transit-gateway-and-aws-cloud-wan/)  
[AWS Cloud WAN Quotas - AWS Network Manager](https://docs.aws.amazon.com/network-manager/latest/cloudwan/cloudwan-quotas.html)  
[Network maximum transmission unit (MTU) for your EC2 instance - Amazon Elastic Compute Cloud](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html)  
[Amazon VPC Transit Gateways Quotas - Amazon VPC](https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html)

PMTUD on Transit Gateway

Add your answer

Relevant content