A customer is building a multicast overlay on AWS using GRE. The general flow is On-Prem1 <-> us-west-2 <-> us-east-1. The VPC in us-west-2 has an IGW and an AL instance. The AL instance is launched in a public subnet with an EIP. The us-west-2 VPC is peered with us-east-1 VPC and the us-east-1 VPC has an AL instance running as well. Creating a GRE tunnel from us-west-2 <-> us-east-1 using the private IPs works fine.
To test the connection back to on-prem we launched a separate VPC in us-west-2 so the flow would be VPC1 -> IGW1 -> IGW2 -> VPC2 but the GRE tunnel will not come up in this scenario. I have updated the security groups to allow ALL TRAFFIC to pass to and from each of the EIPs. I can ping through using the public IPs. After I configure the GRE tunnel, I try to ping the inner IP on the far end, I see it leave the GRE interface but I never see that packet arrive on the far end. Similarly, i never see any GRE 'keep alive' traffic on either interface using these when i capture only the gre proto. I have disabled the SRC/DST check on all of the instances involved here. If I peer the two VPCs and use the private IPs instead of the EIPs, the GRE tunnel comes up just fine. If i force the traffic out the IGW, the tunnel will not come up.
us-west-2 VPC 1
ONBOOT=yes
DEVICE=gre2
TYPE=GRE
MY_INNER_IPADDR=xx.xx.xx.4
MY_OUTER_IPADDR=xx.xx.xx.179
PEER_INNER_IPADDR=xx.xx.xx.3
PEER_OUTER_IPADDR=xx.xx.xx.183
us-west-2 VPC 2
ONBOOT=yes
DEVICE=gre2
TYPE=GRE
MY_INNER_IPADDR=xx.xx.xx.3
MY_OUTER_IPADDR=xx.xx.xx.183
PEER_INNER_IPADDR=xx.xx.xx.4
PEER_OUTER_IPADDR=xx.xx.xx.179
What am i doing wrong? How can i get a GRE tunnel up across the two IGWs?