Connection to on-prem not working when both site-to-site VPN is active

Hi Abdelrahman Sheham,

Please Go through the below approaches, i hope it will help to solve your issue.

1. Manually Disabling a Tunnel via the AWS Console:

AWS provides redundancy for VPN connections by allowing two tunnels (Tunnel 1 and Tunnel 2) between your on-premises environment and AWS. However, using both tunnels simultaneously can sometimes cause routing conflicts, especially if your on-premises routing equipment isn’t configured to handle this redundancy effectively.

While AWS does not offer a direct "disable tunnel" option via the console, you can control which tunnel is active by configuring your on-premises device (such as a router or firewall) to only route traffic through one tunnel. This means you effectively disable the other tunnel by not using it.

Steps:

On-Premises Configuration:

• Access your on-premises VPN device.
• Modify the routing or tunnel settings to ensure that only one of the tunnels is used for the connection to AWS.
• You might want to disable BGP (if used) or static routing for one tunnel while leaving it enabled for the other.
• This ensures that even though both tunnels are technically up, only one is actively used for data traffic.

Monitoring in AWS Console:

• Go to the VPC Console.
• Navigate to VPN Connections and select your VPN connection.
• Under Tunnel Details, you can observe the status of the tunnels. Even if both show as active, only the configured one on your on-premises device will route traffic.

Pros:

• No need for changes on the AWS side, ensuring that your VPN connection remains robust.
• Simple to execute if you are familiar with your on-premises VPN device.

Cons:

• Requires manual intervention to switch tunnels.
• Still dependent on the on-premises device, so it may not be the most flexible or automated approach.

2. Using Static Routes for Failover:

Failover using static routes is a more elegant solution for ensuring redundancy while preventing both tunnels from causing conflicts. Here’s how it works:

Concept:

• You can define static routes on your on-premises device with different priorities, so one tunnel is preferred over the other.
• The higher-priority route is used by default, and the secondary route is only used if the primary one fails.

Steps:

Define Static Routes on Your On-Premises Device:

• Access your VPN device.
• Set up a static route for your AWS VPC CIDR block that directs traffic through Tunnel 1.
• Assign a lower priority (e.g., a higher metric value) to Tunnel 2. This setup ensures that Tunnel 2 only becomes active if Tunnel 1 fails.

AWS Route Table Configuration:

• Within the AWS VPC Console, ensure that the route table associated with the VPN connection is configured to send traffic to your on-premises network through the VGW (Virtual Private Gateway).
• Ensure that there are no conflicting routes that might cause AWS to send traffic through both tunnels simultaneously.

Pros:

• Automated failover without manual intervention.
• Ensures seamless transition from Tunnel 1 to Tunnel 2 in the event of failure.

Cons:

• Requires careful planning and testing to ensure that the failover works as expected.
• Depending on your network's complexity, might involve configuring multiple static routes, which can be complex.

3. BGP Route Propagation for Dynamic Routing:

If you’re using BGP (Border Gateway Protocol) for dynamic routing, you can control which tunnel is preferred based on routing policies. BGP allows you to automatically handle failover without needing to configure static routes manually.

Concept:

• BGP allows for dynamic route updates, where the preferred route can be determined by metrics such as the local preference or AS path length.
• By configuring BGP on your on-premises router, you can ensure that one tunnel is preferred, while the other serves as a backup.

Steps:

Configure BGP on Your On-Premises Router:

• Set a higher local preference or a shorter AS path for the route associated with Tunnel 1.
• Set a lower local preference or a longer AS path for Tunnel 2.

AWS Side Configuration:

• Ensure that your VGW (Virtual Private Gateway) is configured for BGP routing.
• Confirm that the BGP session with AWS is established correctly for both tunnels.

Pros:

• Fully automated and dynamic routing ensures the best path is always chosen.
• Handles failover seamlessly without manual intervention.

Cons:

• Requires a good understanding of BGP and dynamic routing protocols.
• Complexity increases with the size of the network.

4. Scripted Automation Using AWS CLI or SDK:

If you prefer to automate the process of enabling/disabling tunnels, you can use AWS CLI or SDK to script this functionality.

Concept:

• You can automate the process of disabling one of the tunnels by modifying the VPN connection configuration via script.
• This approach is useful if you want to automate failover or if you need to perform regular testing.

Steps:

Install and Configure AWS CLI:

• Ensure AWS CLI is installed on your management system and is configured with appropriate credentials.

Write a Script to Modify VPN Connection:

• Use the aws ec2 modify-vpn-connection-options command to modify the tunnel options.
• The script can disable the BGP session or remove the static route associated with the tunnel you want to disable.

Example CLI Command:

aws ec2 modify-vpn-connection-options --vpn-connection-id vpn-12345678 --tunnel1-options <json-options> --tunnel2-options <json-options>

You can adjust the options to effectively disable the tunnel by making it non-preferred.

Automate the Script:

• Schedule the script to run based on specific conditions or as part of your CI/CD pipeline.

Pros:

• Offers full control and automation over the VPN tunnels.
• Flexible and can be integrated into larger automation workflows.

Cons:

• Requires scripting knowledge.
• Needs regular maintenance and testing to ensure it works as intended.

5. Route Table Configuration for Selective Routing:

If you prefer to manage which tunnel is active directly through the AWS route tables, you can achieve this by adjusting the routes associated with your VPN connection.

Concept:

• Modify the route table associated with your VPC to prefer one tunnel over another by adjusting the routes.

Steps:

Identify the Route Table:

• In the AWS VPC Console, find the route table associated with your VPN connection.
• Check the current routes that direct traffic to your on-premises CIDR block.

Adjust the Routes:

• Modify the route to ensure that traffic is sent only through one of the tunnels.
• You might need to adjust the route propagation or manually set up a static route.

Failover Handling:

• If the primary tunnel fails, manually or automatically adjust the route table to switch traffic to the secondary tunnel.

Pros:

• Direct control over traffic routing.
• Can be integrated with other AWS features, like CloudWatch alarms, to trigger failover.

Cons:

• Requires manual intervention if not automated.
• Might be complex depending on the number of routes and the overall network design.