Extending Layer 2 Networks into VMware Cloud on AWS using L2VPN with NSX Autonomous Edge

Introduction

This post will guide you through how to extend layer 2 networks into VMware Cloud on AWS using layer 2 VPN (or L2VPN) with NSX Autonomous Edge, including a High Availability (HA) pair setup.

VMware Cloud on AWS offers one of the fastest ways to migrate on-premises vSphere workloads to the cloud, while minimizing migration complexity and risks.

Customers can create layer 2 extension (L2E) segments from on-premises data centers into VMware Cloud on AWS. This allows you to retain virtual machine (VM) IP addresses and eliminate the need of reconfiguring application networking during migrations. This significantly simplifies cloud migration process, and helps accelerate dynamic/bursting access to cloud computing resources.

A common method for extending layer 2 networks is to leverage the VMware HCX migration tool, which is provided to VMware Cloud on AWS customers at no additional cost. (Refer to this AWS re:Post article to learn how to create L2E segments using HCX.) However, HCX can only extend vSphere Distributed Switch (vDS) port groups, and it does not support extending vSphere Standard Switch (vSS) port groups.

For customers still running vSS (or without vSphere Enterprise Plus licenses) at on-premises data centres, an alternative L2E method is to deploy L2VPN using the NSX Autonomous Edge. In this post, I will provide a detailed walkthrough to show you how to deploy L2VPN and extend segments into VMware Cloud on AWS using the NSX Autonomous Edge.

 

Pre-requisites

• 1x VMware Cloud on AWS SDDC (I’m running a SDDC on 1.22v5).
• 1x On-premises vSphere cluster (min ver. 6.7U2+ required if running hybrid migration using vMotion, as per VMware KB56991).
• AWS Direct Connect or Internet connectivity between on-premises and SDDC.
• UDP 500/4500 and ESP (IP protocol) are allowed from on-premises NSX Autonomous Edge client to the NSX Edge T0 at VMware Cloud on AWS.
• Download a compatible version of the NSX Autonomous Edge for on-premises deployment, as per the latest L2VPN interoperability matrix. (I’m using NSX Edge 4.0.1.1 for SDDC v1.22)
• Create a L2 Trunk port group (for bridging local L2E VLANs) and bind it to the vSwitch on your hosts. Ensure this port group accepts promiscuous mode and forged transmits.

 

Lab Overview

The above diagram provides an overview of the demo lab setup. For this example, I will deploy a HA pair of the NSX Edge appliances at on-premises, in order to build a L2VPN into the lab SDDC. The L2VPN tunnel is terminated between the on-premises NSX Autonomous Edge (Active) and the SDDC NSX Edge T0 (Active). I will then extend 2x VLANs (VLAN-11 & VLAN-12) from the local vSphere cluster into the VMware Cloud on AWS environment.

For better illustration, below is an overview of the interface and IP address configurations for the NSX Edge pair.

Interface	Description	Primary Edge	Secondary Edge
eth0	Management	192.168.100.20/24	192.168.100.21/24
eth1	Uplink	192.168.101.20/24	n/a, inherit primary node IP during tunnel failover
eth2	L2 Trunk	n/a	n/a
eth3	HA	169.254.100.1/30	169.254.100.2/30

 

Setup

For this example, I’ll walk through the configurations covering the following aspects:

1. Configure a L2VPN with L2E segments at VMware Cloud on AWS
2. Deploy NSX Autonomous Edge at on-premises vSphere cluster
3. Configure L2VPN and network extension using NSX Autonomous Edge
4. Deploy a secondary NSX Edge for HA clustering with a failover test

 

Part1: Configure a L2VPN at VMware Cloud on AWS

To begin, we’ll first setup a L2 VPN session from VMware Cloud on AWS side. Log into the VMware Cloud console, navigate to your SDDC and launch the NSX Manager. Create a L2VPN session under Networking/Network Services > VPN > SDDC > Layer 2. Since I’m using Internet as the VPN underlay, I have selected SDDC local Public IP for the VPN endpoint. You’ll also need to enter the remote public IP for the on-premises NSX edge client, and make sure you provide its private IP if the appliance is going to sit behind a NAT device.

Once the L2VPN session is configured, use the provided link to download the OVA of the NSX Autonomous Edge appliance if you haven’t already done so. Also, you’ll need to download the VPN config file here which contains the peer_code that is required for setting up the on-premises NSX Edge at a later stage.

Under the VPN section, we will add two L2E segments to be mapped to VLAN-11 & VLAN-12 at the on-premises vSphere network. You’ll need to provide a unique Tunnel ID for each L2E segment, and it needs to match the remote Tunnel ID configured at the on-premises NSX Edge.

 

Part2: Deploy NSX Autonomous Edge at on-premises

We will now deploy the primary NSX Autonomous Edge appliance at the on-premises vSphere cluster. Deploy the OVA template and select the interface layout for the NSX appliance as per below.

At the “customize template” section, select the “Autonomous Edge” mode and configure the Management interface (eth0) settings.

Continue to the DNS and NTP configurations, and enable/disable SSH access as per your requirements.

Next, configure network settings for the External/Uplink interface (eth1) and the HA interface (eth3). Note you’ll need to provide VLAN IDs here if the attached vSwitch port groups are in trunk mode. In my case, for both interfaces the VLANs are tagged at the port groups so I put “0” as the VLAN ID here. I will leave the HA Port default gateway empty as I’m using a dedicated /30 for the HA subnet. Since this is the primary node, we’ll leave out all secondary-node related HA configurations. Go ahead to deploy and power on the appliance.

 

Part3: Configure L2VPN with L2E using NSX Autonomous Edge

Once the NSX Edge appliance is online, log into the management console using the Admin account. First, go to L2VPN to add a session, you’ll need the Peer Code captured from the SDDC L2VPN configuration file from Part-1.

You should see the L2 VPN session coming up on both sides in a few seconds.

Now switch to the PORT configuration, we’ll create two ports for bridging to the local VLAN-11 and VLAN-12 --- note the Exit interface is the L2 Trunk port group (eth2).

Next, go back to L2VPN section to attach the two local ports to the L2VPN session. For each segment we’ll need to keep the same Tunnel ID to the SDDC side.

At this stage we could run a quick test to verify the L2VPN connectivity. As shown below, an on-premises VM (10.10.11.102) on VLAN-11 can indeed reach a remote VM (10.10.11.101) over the extended segment on SDDC via the L2 VPN.

 

Part4: Deploy secondary NSX Edge for HA clustering

Now we will deploy a secondary NSX Edge appliance to create a HA cluster. First, we’ll SSH into the primary Edge using the Admin account, and retrieve its API thumbprint using the command:

get certificate api thumbprint

Repeat the same process in Step-2 to deploy a secondary NSX Edge appliance. Make sure to tick the “Secondary API Node” option, and you’ll need to supply the management IP and credentials of the primary Edge, as well as its thumbprint captured from above.

Once the secondary appliance comes online, it will automatically sync all configurations with the primary node and form an Active/Standby HA pair.

To verify the NSX HA cluster, power off the primary Edge and we can see the L2E traffic resumes within a few seconds. In my case, it dropped only 5 pings before the L2VPN tunnel fails over to the secondary appliance.

Back at the management console of the secondary Edge, we can see its HA status is now switched to Active status.

 

Additional Considerations

• Prior to v1.18, each SDDC is limited to a single L2VPN tunnel with a maximum of 16x L2E Segment/VLAN (via the NSX Autonomous Edge). From v1.18 onwards, you can now deploy customized T1 Gateways and create additional L2VPN tunnels directly into the T1s.
• When running L2E segments via L2VPNs, you can use either VMware HCX or Hybrid Migration with vMotion for workload migrations into VMware Cloud on AWS.
• Unlike HCX, NSX Autonomous Edge does not provide L2E traffic optimization like the Mobility Optimized Network (MON). To eliminate traffic hairpinning and reduce latency, it is recommended that you cutover the L2E segments onto the SDDC side as soon as migration completes.  
  
  

Conclusion

In this article I have walked through an example of how to extend layer 2 networks into VMware Cloud on AWS using L2VPN with NSX Autonomous Edge.