Why is Windows MTU set to 9001 for some instances and 1500 for others ?

0

I noticed that booting Windows 2019 Base AMI on a:

  • t3.large (Nitro-based) resulted in having "Ethernet 3" configured with a 1500 bytes MTU.
  • t2.large (Xen-based) resulted in having "Ethernet" configured with a 9001 bytes MTU.
EXPERT
asked 3 years ago1303 views
1 Answer
1
Accepted Answer

As of the time of posting, Windows may use any of 3 different network adapters (and corresponding drivers) on EC2 depending on the the instance type in use and whether or not Enhanced Networking is available/enabled. Loosely, these are as follows:

  • AWS PV: All Xen types where Enhanced Networking is not available or not enabled
  • Intel 82599VF: Some Xen types (C3, C4, D2, I2, M4 excluding m4.16xlarge, and R3) where Enhanced Networking (SriovNetSupport : Simple) is enabled
  • ENA : All Nitro instance types (C5, M5, R5, T3, etc) and a few Xen types (R4, M4.16xlarge, i3) where Enhanced Networking (EnaSupport : True) is enabled

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/enhanced-networking.html

MTU is not configured to a specific value on AWS provided Windows AMIs, but rather, the default configuration for each driver is applied. However, as the drivers are provided by different vendors/teams, they behave a bit differently.

In the example provided, Windows on t3.large will use the ENA driver and network adapter, while on t2.large, AWS PV is used. AWS PV will automatically enable jumbo frames if it detects support by the instance type (most all current gen instance types). However, ENA does not automatically adjust this setting, and is configured to 1500 MTU by default. Likewise, the Intel 82599VF driver does not attempt to automatically configure settings and is also set to 1500 MTU.

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/xen-drivers-overview.html

Jumbo frames will get fragmented on exit from VPC, which Windows doesn't always deal with exceptionally well. As an optimized configuration is relative on the work being done, guidance is made available to customers in the public docs regarding discovery of path MTU and configuration per adapter. Additionally, more OS config optimization guidance for ENA is available, which is where the primary development focus is today.

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/network_mtu.html

https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/enhanced-networking-os.html

AWS
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions