How do I troubleshoot Direct Connect SiteLink connectivity issues?

Resolution

To resolve SiteLink connectivity issues, check the SiteLink configuration, troubleshoot routing, and traffic flow. The following troubleshooting steps also address issues that prevent you from receiving remote site prefixes from Border Gateway Protocol (BGP).

Important: Be sure to perform checks on both the AWS side and on the customer gateway.

For general Direct Connect troubleshooting, including layers 1, 2, and 3, see Troubleshoot AWS Direct Connect.

Check the SiteLink prerequisites and configuration

On the AWS Management Console

To check your SiteLink confirmation, complete the following steps:

1. In the Direct Connect console.
2. Select each virtual interface, and then confirm that SiteLink shows as Enabled.
3. Check that SiteLink uses a supported virtual interface type.
4. For a list of supported interface types, see the virtual interfaces table in the SiteLink section of Direct Connect virtual interfaces and hosted virtual interfaces
5. Check that you attached all your virtual interfaces to the same Direct Connect gateway. Virtual interfaces that use SiteLink must share the same Direct Connect gateway to communicate.
6. Check that the state of your virtual interface is Available and the status of BGP is UP.
7. Confirm the AWS Region of the virtual interface that SiteLink must use.
   Note: SiteLink isn't available in AWS GovCloud (US) or China Regions.

On the customer gateway

Note: The following command line interface (CLI) commands use Cisco IOS syntax. Actual commands vary depending on your router vendor, operating system, and software version. Refer to your device documentation for equivalent commands.

On the customer gateway, check that the state of the BGP session is Established, run the following show ip command:

show ip bgp neighbors Amazon_peer_IP

Note: Replace Amazon_peer_IP with your Amazon peer IP address.

To confirm that your device advertises prefixes and receives routes from the remote virtual interface that uses SiteLink, run the following show ip commands:

show ip bgp neighbors Amazon_peer_IP advertised-routes

show ip bgp neighbors Amazon_peer_IP received-routes

Note: Replace Amazon_peer_IP with your Amazon peer IP address.

For general BGP troubleshooting steps, see How can I troubleshoot BGP connection issues over Direct Connect?

Check routing and BGP advertisements

On the AWS Management Console

Check the following Amazon CloudWatch metrics for each virtual interface:

• If the VirtualInterfaceBgpPrefixesAccepted metric is 0, then AWS doesn't accept routes from your device. Confirm that your device correctly advertises prefixes.
• The VirtualInterfaceBgpPrefixesAdvertised metric includes routes from other virtual interfaces that use SiteLink on the same Direct Connect gateway. SiteLink correctly functions when the value of this metric is higher than the number of prefixes that you advertise locally.
• If the value of the VirtualInterfaceBgpStatus metric is 1, then your BGP is UP.

For metric definitions, see Monitor with Amazon CloudWatch.

Use the console to view your virtual interface details. Navigate to the BGP peers section to inspect the prefix allocations for your sites. Be sure that each of your sites advertises unique prefixes or uses AS path prepending.

Check that your connection doesn't exceed its prefix quota. If your connection exceeds its prefix limit quota, then AWS ends your BGP session.

Note: SiteLink has a separate 1,000-prefix quota from the 100 routes quota for each BGP session.

To recover your BGP session, reduce the number of advertised prefixes to meet the quota, and then reset the BGP session from your device.

On the customer gateway

Note: The following CLI commands use Cisco IOS syntax. Actual commands vary depending on your router vendor, operating system, and software version. Refer to your device documentation for equivalent commands.

On the customer gateway, run the following show ip command to confirm that your device receives remote site prefixes:

show ip bgp neighbors Amazon_peer_IP received-routes

Note: Replace Amazon_peer_IP with your Amazon peer IP address.

If you don't see remote site prefixes, then confirm that the remote site router advertises its prefixes to AWS. Check that both virtual interfaces use SiteLink. Confirm that you attached both virtual interfaces to the same Direct Connect gateway.

Check that the AS_PATH includes the Autonomous System Number (ASN) of the Direct Connect gateway on AWS and the remote site's customer ASN.

On the customer gateway, run the following show ip command to check the configuration of the routing table:

show ip route remote_site_prefix

Note: Replace remote_site_prefix with the prefix of the remote site you must connect to.

In the output, confirm that you installed the remote site prefix as an active BGP route through your Direct Connect virtual interface.

Check traffic flow and physical connectivity

On the AWS Management Console

Complete the following steps:

1. Monitor the following traffic metrics in CloudWatch:
   VirtualInterfaceBpsEgress
   VirtualInterfaceBpsIngress
   VirtualInterfacePpsEgress
   VirtualInterfacePpsIngress
2. Check the following physical connection state metrics in CloudWatch:
   A ConnectionState value of 1 shows that the connection is up, 0 shows that the connection is down.
   High ConnectionErrorCount values show physical layer issues such as cyclic redundancy check (CRC) errors.
   Note: To troubleshoot your layers, see Troubleshoot layer 1 (physical) issues and Troubleshoot layer 2 (data link) issues.

On the customer gateway

Note: The following CLI commands use Cisco IOS syntax. Actual commands vary depending on your router vendor, operating system, and software version. Refer to your device documentation for equivalent commands.

Check maximum transmission unit (MTU) configuration. SiteLink supports jumbo frames. For details, see MTUs for private virtual interfaces or transit virtual interfaces. SiteLink supports the following maximum values:

• Transit virtual interface has a maximum of 8,500 bytes.
• Private virtual interface has a maximum of 9,001 bytes.

To test your configuration, run the following ping command:

ping remote_site_IP size 8000 df-bit

Note: Replace remote_site_IP with the IP address of the remote site. Use a packet size close to your configured MTU. Be sure to subtract 28 bytes for IP/Internet Control Message Protocol (ICMP) headers.

To check the traffic flow and connectivity on your customer gateway, complete the following steps:

1. Check that your network access control lists (network ACLs) and firewall rules allow traffic to and from the remote site prefixes.
2. Check virtual local area network (VLAN) tagging. The 802.1Q VLAN ID must match the AWS console configuration.
3. If large packets fail but small packets succeed, then check MTU on all devices in the path.

Resolve unexpected path selection

If traffic takes an unexpected path or you don't see prefixes where you expect them, then Check AS_PATH-based path selection. When you activate SiteLink, AWS prefers the path with the shortest AS_PATH regardless of the associated Region.

Check whether the remote virtual interface has deactivated SiteLink. The Direct Connect gateway advertises prefixes from a virtual interface that has deactivated SiteLink only to gateway associations. It doesn't advertise prefixes to other virtual interfaces where you activated SiteLink. If you don't see expected prefixes from a remote site, then confirm that you activated SiteLink on that remote virtual interface.

Check for duplicate prefix advertisements. If you advertise the same prefix on multiple virtual interfaces that use SiteLink, then traffic takes an unexpected path, or the prefix doesn't propagate correctly. Make sure that each location advertises unique prefixes. If you must advertise the same prefix from different locations, then use Autonomous System (AS) path prepending. For instructions, see the Routing and forwarding behavior when using SiteLink section of Advanced Routing scenarios with AWS Direct Connect SiteLink.

Be sure to use unique ASNs for each site or configure BGP knobs. For more information, see the Customer BGP Autonomous System Number (ASN) consideration section of Introducing AWS Direct Connect SiteLink.

For more information, see Influence Direct Connect traffic to on-premises.