How do I use BGP logs to troubleshoot why my AWS VPN tunnel went down?

Resolution

Activate VPN tunnel logging

Important: BGP logs are available only for events that occur after you activate logging. If the tunnel flaps before you activate BGP logs, then those events don't appear in the logs.

Complete the steps in the To enable tunnel BGP logging on an existing Site-to-Site VPN connection section of Enable AWS Site-to-Site VPN logs.

After you activate logging, BGP events appear in your Amazon CloudWatch log group as the --BGP.log log stream.

Understand BGP log format

BGP logs have two message types, BGPStatus or RouteStatus messages.

Review the BGPStatus messages to see session state changes and errors. The Status field shows UP or DOWN. The message.details field contains diagnostic text, such as "Hold Timer Expired" or "maximum prefix limit was exceeded".

Review RouteStatus messages to see route advertisements and denials. The Status field shows ADVERTISED for accepted routes or UPDATED for changed or denied routes. The message object includes the prefix, asPath, nextHopIp, and weight fields.

Use the timestamp field to correlate BGP events with VPN tunnel activity logs and CloudWatch TunnelState metrics.

Identify the BGP failure pattern

Review the BGP log entries in Amazon CloudWatch Logs to determine why sessions fail. Use your findings to resolve issues that you encounter.

Resolve the "Maximum prefix limit was exceeded" log message

The maximum prefix quota for VPNs with Virtual Private Gateways is 100. The quota for VPNs with AWS Transit Gateways is 1,000. If the customer gateway advertises more route prefixes than the quota, then you receive a log message similar to the following:

"AWS-side peer detected the maximum prefix quota was exceeded - received 1001 prefixes from neighbor 169.254.#.#, limit is 1000"

Customer gateways that advertise more prefixes than the allowed quota cause BGP sessions drop from Established to Idle. Then, the status of your VPN tunnel changes from UP to DOWN.

To resolve, reduce the number of prefixes that you advertise from your customer gateway. Summarize, or aggregate routes to combine specific routes into larger CIDR blocks, such as to replace 256 individual /24 routes with a single /16 route.

If you reach 75% of your maximum prefix quota, then you receive a BGP status message similar to the following example in your BGP logs:

"AWS-side peer is reporting a maximum prefix limit warning"

To resolve, create a CloudWatch alarm for this message to detect prefix growth before it reaches the quota. For more information, see the BGP Status Messages section of Site-to-Site VPN log contents.

Resolve the "Hold Timer Expired" log message

The default hold time is 30 seconds. If the AWS-side peer didn't receive BGP keepalive messages within the negotiated hold time, then you receive a log message similar to the following:

"AWS-side peer sent a Hold Timer Expired notification to neighbor 169.254.#.#"

This log message shows network connectivity issues between the customer gateway and the VPN endpoint. You receive this log message for connectivity issues that can include transient path failures, high packet loss, and firewalls that block BGP traffic.

Customer gateways that don't send keepalive messages within the hold time cause the BGP sessions drop. Also, the status of your VPN tunnel changes from UP to DOWN after a period of connectivity loss.

To resolve, complete the following steps:

1. Check your customer gateway device for interface errors, high CPU, or packet drops.
2. Confirm that a firewall, access list, or security group doesn't block TCP port 179 or the tunnel inside addresses in the 169.254.x.x/30 range.
3. Review your VPN tunnel's packet loss metrics in CloudWatch.
4. If the issue is intermittent, then check for Dead Peer Detection or Internet Key Exchange (IKE) rekeying conflicts on your customer gateway. For instructions, see How do I troubleshoot Site-to-Site VPN tunnel inactivity, tunnel flapping, or down tunnel on my customer gateway device?

Resolve the "Bad Peer AS" issues caused by ASN mismatch

If the customer gateway received an Autonomous System Number (ASN) that doesn't match its expected value during the BGP OPEN exchange, then you receive a log message similar to the following:

"AWS-side peer received notification 2/2 OPEN Message Error/Bad Peer AS from neighbor 169.254.#.#"

For example, if you configured the customer gateway with remote-as 65001 but the AWS side advertises AS 64512, then you see the preceding log message.

If you misconfigured a customer gateway with an incorrect remote-as value, then the BGP never reaches the Established state. Also, the session cycles between Connect and Active.

To resolve, confirm that the following remote-as values on your customer gateway match the ASN assigned to the AWS side:

• For Virtual Private Gateways, the ASN is the default value of 64512 or the custom ASN that you specified when you created the Virtual Private Gateway.
• For AWS Transit Gateway, the ASN exists in the AmazonSideAsn property of the Transit Gateway.

Resolve the "DENIED due to: as-path contains our own AS" log message

If BGP loop prevention rejects routes with Autonomous System (AS) paths that contain the ASN of the receiving router, then you receive a log message similar to the following:

"DENIED due to: as-path contains our own AS;"

This error occurs when the route traverses a network loop and returns to the AS that originates the route. It also occurs when the customer gateway prepends the ASN into the AS-path of advertised routes.

If the customer gateway includes the AWS-side ASN in the AS-path, then specific routes don't appear in the VPC route table or TGW route table. However, the BGP session still establishes.

On your customer gateway device, run one of the following BGP route inspection commands for your router to check the AS-path for the rejected routes.

• For Cisco IOS:

  show bgp

• For other platforms:

  show ip bgp

Then, remove route maps or policies that prepend the ASN on the AWS side into your advertisements.

Resolve missing routes that don't have BGP log message

If you don't find BGP log entries that reference a missing prefix, then the customer gateway doesn't advertise a specific route to AWS.

Important: If AWS rejects your route, then the BGP logs show a "DENIED" entry. If you don't see a log message for a prefix, then the AWS VPN endpoint didn't receive the route from your customer gateway.

If customer gateways don't advertise expected routes, then specific routes that you expect don't appear in the AWS route table. The BGP sessions remain Established.

To resolve this error, confirm the following BGP configurations for your customer gateway:

• Confirm that the network statement or route redistribution includes the missing prefix.
• Check that outbound route-maps or prefix-lists configurations filter the route.
• Confirm that a matching route exists in the routing table for the customer gateway.

Related information

View Amazon CloudWatch Logs metrics for AWS Site-to-Site VPN

AWS Site-to-Site VPN quotas

Troubleshooting AWS Site-to-Site VPN customer gateway device