Why is my Site-to-Site VPN BGP session failing or flapping?

Resolution

Identify the BGP state that the session gets stuck in

To turn on tunnel BGP logging, complete the steps in the To enable tunnel BGP logging on an existing Site-to-Site VPN connection section of Enable AWS Site-to-Site VPN logs.

Open your BGP logs in Amazon CloudWatch Logs, and then review the BGPStatus events. Check the event in the Status field. Each event includes a Message field that includes the BGP state transition messages. For the full list of state and error messages, see Site-to-Site VPN log contents.

If you receive a "BGP session state has changed from Idle to Connect" or "From Connect to Active" error message, then the session is stuck in the Connect or Active state. To resolve this error, complete the steps in the Resolve a BGP session that's stuck in the Connect or Active state section of this article.

If you receive a "BGP session state has changed from OpenConfirm to Established" error message with a "From Established to Idle" message, then the session flapped.

Note: An Active BGP state doesn't show that the status of your session is UP. It shows that BGP is retrying the TCP connection and hasn't reached the Established state.

To resolve a flapping connection, or a BGP session that doesn't establish, complete the steps in the Resolve a BGP session that flaps between UP and DOWN section of this article.

Resolve a BGP session that's stuck in the Connect or Active state

Download the VPN configuration file in the Amazon Virtual Private Cloud (Amazon VPC) console. Confirm that your BGP neighbor's statement uses the Virtual Private Gateway or AWS Transit Gateway inside tunnel IP address in the 169.254.0.0/16 range. Don't use the public gateway endpoint IP address as the BGP neighbor. For more information, see Download the configuration file.

From the customer gateway tunnel interface, ping the AWS inside tunnel IP address to confirm reachability between peers. If the pings fail, then the Internet Protocol Security (IPsec) tunnel isn't up, or the tunnel interface routing is misconfigured.

Then, to confirm that Internet Key Exchange (IKE)/Phase 1 and IPsec/Phase 2 establish, open the Amazon VPC console. On the Tunnel Details tab, in the Details column, check that the statuses of IKE/Phase 1 and IPsec/Phase 2 are UP. If the status of IKE/Phase 1 is DOWN, then see Why is IKE (phase 1 of my VPN tunnel) failing in Amazon VPC? If the status of IPsec is DOWN, then see Why is my Site-to-Site VPN down?

Check the access control lists and firewall rules on the customer gateway tunnel interface for TCP port 179. Run a packet capture on the customer gateway to confirm that TCP SYN packets for port 179 arrive from the AWS inside tunnel IP address.

Resolve the "Detected bad OPEN message" log message

The remote Autonomous System Number (ASN) in your device's BGP configuration must match the ASN on the AWS side. If the ASNs don't match, then you receive a log message similar to the following:

"Detected a bad OPEN message - remote AS is X, expected Y"

To resolve this error, follow the instructions in the Resolve the "Bad Peer AS" issues caused by ASN mismatch section of How do I use BGP logs to troubleshoot why my AWS VPN tunnel went down?

Resolve a BGP session that flaps between UP and DOWN

Resolve the "Hold Timer Expired" log message

The AWS default hold time is 30 seconds. If the AWS side peer didn't receive BGP keepalive messages within the negotiated hold time, then you receive a log message similar to the following:

"AWS-side peer sent a Hold Timer Expired notification to neighbor 169.254.#.#"

To resolve this error, follow the instructions in the Resolve the "Hold Timer Expired" log message section of How do I use BGP logs to troubleshoot why my AWS VPN tunnel went down?

Resolve the "Administrative Reset" log message

If the customer gateway resets the BGP session and the session drops and re-establishes, then you receive a log message similar to the following:

"Received notification 6/4 (Cease/Administrative Reset)"

To resolve the issue, check the customer gateway logs for configuration changes, policy updates, device reloads, or automation that clears BGP sessions.

Resolve the "Maximum Number of Prefixes Reached" log message

Virtual private gateways allow a maximum of 100 prefixes advertised from your customer gateway. Transit gateways allow a maximum of 1,000 prefixes advertised from a customer gateway. If your customer gateway advertises more prefixes than the limit, then you receive a log message similar to the following:

"Notification 6/1 (Cease/Maximum Number of Prefixes Reached)"

The BGP session drops from Established to Idle, and the status of your VPN tunnel changes from UP to DOWN.

To resolve this error, follow the instructions in the Resolve the "Maximum prefix limit was exceeded" log message section of How do I use BGP logs to troubleshoot why my AWS VPN tunnel went down?

For more information, see Site-to-Site VPN quotas.

Resolve the "Other Configuration Change" log message

If AWS tunnel endpoint maintenance occurs, then you receive a log message similar to the following:

"Notification 6/6 (Cease/Other Configuration Change)"

This error shows that the affected tunnel drops during the maintenance event. To resolve the issue, check the AWS Health Dashboard for scheduled maintenance events. Then, configure both tunnels with automatic failover so that sessions recover automatically. For more information, see Site-to-Site VPN tunnel endpoint replacements.

Related information

Troubleshooting AWS Site-to-Site VPN

Customer gateway device best practices