vpn tunnel traffic change

Thank you for your detailed question about AWS Site-to-Site VPN behavior. I'll address your concerns and questions one by one:

1. AWS Site-to-Site VPN is designed to be active/active by default. This means that both tunnels can be used simultaneously for traffic, providing redundancy and load balancing. When both tunnels are up, AWS can use either tunnel for traffic.

2. When you modify a VPN tunnel, it does impact the traffic flow. During the modification process, the tunnel being modified becomes unavailable. This unavailability causes the traffic to shift to the other available tunnel.

3. Yes, active tunnels can change during AWS's regular tunnel replacements. AWS periodically updates tunnel endpoints for various reasons, including upgrades and hardware retirement. These updates are applied to one tunnel at a time to maintain connectivity. During this process, traffic will shift to the unaffected tunnel.

4. The behavior you're observing is the expected failover mechanism of AWS Site-to-Site VPN. When you modify tunnel 1, it becomes temporarily unavailable, causing all traffic to flow through tunnel 2. Once tunnel 1 is back up and available after the modification, AWS may route traffic back to it, especially if it's the preferred path based on your routing configuration.

It's important to note that when you modify VPN tunnel options, connectivity over the modified tunnel is interrupted for up to several minutes. This interruption causes the traffic to shift to the other available tunnel. Once the modification is complete and the tunnel is back up, AWS may route traffic back to it if it's the preferred path.

The behavior you're seeing with traffic flowing back to tunnel 1 after modification could be due to your routing configuration, including the priority settings you mentioned on your Fortigate equipment. If tunnel 1 is set as the preferred path, traffic will naturally flow back to it once it's available again.

To ensure high availability and smooth failover, it's crucial to configure both tunnels properly. AWS strongly recommends configuring both tunnels for redundancy. If one tunnel is down for more than an hour in a day, you'll receive a notification through the AWS Health Dashboard.

Remember to plan for expected downtime when modifying tunnel options and always ensure that both tunnels are properly configured for optimal performance and reliability.
Sources
AWS Site-to-Site VPN tunnel endpoint replacements - AWS Site-to-Site VPN
AWS Health and AWS Site-to-Site VPN events - AWS Site-to-Site VPN
Modify AWS Site-to-Site VPN tunnel options - AWS Site-to-Site VPN
Configure VPN tunnel options | AWS re:Post

To add some context here, the expected behavior is dependent on the type of gateway you are using [Transit Gateway (TGW) or Virtual Private Gateway (VGW)] and whether you have deployed a static routing vpn or dynamic/BGP vpn. Please do have a look at this article here which details the various possible scenarios and hopefully this will help explain what you are observing as well as provide you with a way forward to achieve a consistent behavior.