AWS DX Active/Active on Cisco ASA - Asymmetric traffic problem

Question

Hello,

I'm a newbie when it comes to BGP and I'm having an issue where I'm trying to introduce a second AWS DX to an existing single AWS DX config on my Cisco ASA and whenever I enable to second BGP neighbor, I get sort of asymmetric traffic?

Let's say I'm pinging 5 different EC2's from a server behind the Cisco ASA. As soon as I enable the second BGP neighbor, 3 of those EC2's will stop pinging and the other two continue to ping, then if I do a BGP clear, some other EC2 stops pinging and some start to ping again. It seems like the ASA doesn't like or know how to return traffic back to AWS when its being load balanced between the two DX's.

Has anyone run into similar issues like that? Here is a snippet of my BGP config on the ASA:
```
route-map LOCAL-PREF-AWS permit 10

set local-preference 200

router bgp 65532
 bgp log-neighbor-changes
 bgp graceful-restart
 address-family ipv4 unicast
  neighbor 169.254.96.1 remote-as 64520
  neighbor 169.254.96.1 password *****
  neighbor 169.254.96.1 fall-over bfd
  neighbor 169.254.96.1 activate
  neighbor 169.254.96.1 weight 500
  neighbor 169.254.96.1 route-map LOCAL-PREF-AWS in
  neighbor 169.254.96.1 route-map LOCAL-PREF-AWS out
  neighbor 169.254.96.9 remote-as 64520
  neighbor 169.254.96.9 password *****
  neighbor 169.254.96.9 fall-over bfd
  neighbor 169.254.96.9 activate
  neighbor 169.254.96.9 weight 500
  neighbor 169.254.96.9 route-map LOCAL-PREF-AWS in
  neighbor 169.254.96.9 route-map LOCAL-PREF-AWS out
```

Did some searching and it appears I may be missing the following command?

```
router bgp 65532
address-family ipv4 unicast
maximum-paths 2
```

Are there any other commands that I may be missing?

Thank you.

Answer

This seems exactly to be an issue of asymmetric routig. As Tushar mentioned stateful firewalls don't like this.
Looking at your configs roughy it seems you are using Local Pref to control egress traffic. You would keep local pref for primary BGP session higher than secondary connection on the received routes.
But I also noticed there is no route map to influence traffic from AWS to ASA. You would need either advertise AS_PATH prepend or BGP Communities to influance path from AWS to ASA and ensure it is same as egress.
https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html#bgp-communities-private-transit

This AWS article discuss how the use of AS_PATH prepend or Communities.
https://repost.aws/knowledge-center/active-passive-direct-connect

This Cisco article shows you an example on how to use communities and create route-maps.
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/28784-bgp-community.html

Here are some good read from cisco on working with BGP. 
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13762-40.html

Answer

Most firewalls are stateful in nature and do not like asymmetric routing, some of the Next Gen firewalls do support asymmetric routing, you can enable it. You would need to check if the firewall supports asymmetric routing, if not you can setup Active/Passive paths.

https://docs.aws.amazon.com/directconnect/latest/UserGuide/routing-and-bgp.html

AWS DX Active/Active on Cisco ASA - Asymmetric traffic problem

Relevant content