EC2 in IPv6 VPC unreachable from outside - why?

Question

Hi all,  
I have an IPv6 capable (t2.small) test server running Ubuntu 16.04.1 set up in the eu-west-1 area. The VPC has an IPv6 allocation, and subnets have IPv6 /64s, too. The NAT Internet Gateway is set up as well an an IPv6 Egress Internet Gateway, both set up in the main route table for the 0.0.0.0/0 and ::/0 prefixes, respectively.  
The network ACLs associated with the subnets allow for both incoming and outgoing traffic from and to ::/0.  
Similarly, the Security Group associated with the instance permits incoming traffic from ::/0 to tcp:80, tcp:443 and any incoming IPv6 ICMP traffic from ::/0.  
The network interface attached to the VM has an permanently assigned IPv6 (2a05:d018:n:n::23) address.  
  
From the running instance, I can see via tcpdump that IPv6 traffic is flowing neatly, e.g.

```
10:56:42.285595 IP6 2a05:d018:n:n::23.123 > 2001:67c:1560:8003::c7.123: NTPv4, Client, length 48
10:56:42.296728 IP6 2001:67c:1560:8003::c7.123 > 2a05:d018:n:n::23.123: NTPv4, Server, length 48
```

However, trying to access the server by its IPv6 address from the Internet - either by means of pinging through a looking glass such as  or by trying to access the website from home (with long-time working, afaik filterless IPv6 connectivity) fails.  
  
A tracepath6 breaks off some perceived hops before the machine:

```
 9:  2600:9000:eee::1c7                                   40.986ms asymm 18 
10:  2a01:578:0:10::c                                     40.118ms asymm 17 
11:  no reply
12:  2a01:578::1                                          40.245ms asymm 14 
13:  no reply
14:  2a01:578::13                                         41.313ms 
15:  2a01:578:0:10::8                                     41.910ms asymm 12 
```

and both ping and http requests do not even show up in the VM's tcpdump.  
  
I'm currently stuck in seeing which option I may have missed, or what causes the connectivity to fail. Any eye-opening assistance would be appreciated.  
  
Best regards  
Dominik

Accepted Answer

OK. I did some tests and here are the results.  
  
I decided to use my Windows 2012 Server instance and found out that IPv6 actually works on that instance. Then I looked at the default route on that Windows machine (which was missing in my Ubuntu instance):

```
\> route print
[…]
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      172.31.16.1     172.31.xx.xx     10
[…]
IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
 12    266 ::/0                     fe80::be:a9ff:fede:cc05
```

Since both of my instances are on the same subnet, I decided that the same gateway IPv6 address may work at my Ubuntu instance also, so I entered the following commands:

```
$ route -A inet6 add default gw fe80::be:a9ff:fede:cc05 eth0

$ route -A inet6 | grep eth0
2600:xx:xx:xx::1:1/128    ::                         U    256 0     0 eth0
fe80::/64                 ::                         U    256 0     0 eth0
::/0                      fe80::be:a9ff:fede:cc05    UG   1   0     0 eth0
ff00::/8                  ::                         U    256 2     0 eth0

$ ip -6 route show
2600:xx:xx:xx::1:1 dev eth0  proto kernel  metric 256
fe80::/64 dev eth0  proto kernel  metric 256
default via fe80::be:a9ff:fede:cc05 dev eth0  metric 1
```

And now everything works: I can reach IPv6 Internet hosts from my instance and my instance can be reached from IPv6 Internet clients. I just put a test CGI script on my IPv6 enabled test site:  
  
http://www.kadifeli.name.tr/cgi/sysinfo.pl  
  
You can see your client and my server IPv6 address towards the end of the script output (REMOTE_ADDR and SERVER_ADDR).  
  
--  
  
DominikN,  
  
I think your problem is that you are using “Egress Only Internet Gateway.” I deleted my Egress Only Internet Gateway and I am currently using just my old “Internet Gateway”. No need for “NAT Internet Gateway” also.  
  
My Route Tables’ Routes are like this:

```
Destination      Target      Status      Propagated
172.31.0.0/16    local       Active      No
2600:xx:xx:xx::/56  local     Active     No
0.0.0.0/0        igw-xxxx    Active      No
::/0             igw-xxxx    Active      No
```

(Note: Same gateway for IPv4 and IPv6!)  
  
Network ACLs ALLOW inbound and outbound traffic for “::/0”  
  
Security Groups for the instances allow “ALL Traffic” for destination “::/0” (Outbound Rules). Inbound Rules are permissive on Protocol / Port / Source bases, like:

```
HTTP (80)         TCP (6)          80    ::/0
HTTPS (443)       TCP (6)          443   ::/0
All ICMP - IPv6   IPv6-ICMP (58)   ALL   ::/0
```

---  
  
Last words:  
I am still confused on why the default IPv6 route is not automatically added in my Ubuntu 14.04.5 LTS instance when eth0 goes up and I have to add it manually…

Answer

Hi fedkad,  
  
thanks for the eye-opening referral to using an igw instead of an eigw. No idea where I took the impression from that I had to use an eigw, but it works like a charm now.  
  
Best regards  
Dominik  
  
PS: Out of curiosity, I'll try to look into the 14.04 issue with IPv6 later, using the Oregon Region, too.

Answer

Currently, I am one step behind of you. I have done everything given in the VPC documentation, but my **t2-micro** Ubuntu (14.04.5 LTS) instance can only access the IPV6 address (2600:xx:xx:xx::1:1) assigned to it:  
  
$ **ping6 2600:xx:xx:xx::1:1**  
...  
( pings OK.. )  
or  
$ **nc -v 2600:xx:xx:xx::1:1 22**  
Connection to 2600:xx:xx:xx::1:1 22 port $$tcp/ssh$$ succeeded!  
SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8  
...  
  
However, the instance cannot access any other IPV6. For example:  
  
$ **ping6 2607:f8b0:400a:800::200e**  
connect: Network is unreachable  
or  
$ **nc -v  2607:f8b0:400a:800::200e 80**  
nc: connect to 2607:f8b0:400a:800::200e port 80 (tcp) failed: Network is unreachable  
  
What am I missing?

Answer

Hi DominikN,  
  
Although my answers to your 3 questions are _yes_, I do _not_ have a default IPv6 route in my instance. Only these:  
  
$ **ip -6 route show**  
2600:xx:xx:xx::1:1 dev eth0  proto kernel  metric 256  mtu 9001  
fe80::/64 dev eth0  proto kernel  metric 256  mtu 9001  
  
I did "ifdown eth0; ifup eth0" or even I rebooted my instance with nothing changing.  
  
_Note:_ IPv4 is functioning normally:  
  
$ **ip  route show**  
default via 172.31.16.1 dev eth0  
172.31.16.0/20 dev eth0  proto kernel  scope link  src 172.31.xx.xx  
  
I am on us-west-2.  
  
_QUESTIONS:_  
1. Is still IPv6 on EC2 instances experimental?  
2. My instance, the VPC, and all related resources (interfaces, subnets, etc.) were created in mid 2015 and have "short" ids. May the problem be related to this?

Answer

Hi fedkad,  
  
I think for 14.04 you will need to configure the /etc/network/interfaces.d/xxx.cfg to explicitly accept ra's:

```
ubuntu@ip-172-31-xx-yy:~$ cat /etc/network/interfaces.d/eth0.cfg 
auto eth0
iface eth0 inet dhcp
iface eth0 inet6 dhcp
accept_ra 1
ubuntu@ip-172-31-xx-yy:~$ ip -6 ro sh
2600:1f14:nn:nn:6d74:6b1b:24f9:65de dev eth0  proto kernel  metric 256 
fe80::/64 dev eth0  proto kernel  metric 256  mtu 9001
default via fe80::483:68ff:fe99:80bd dev eth0  proto ra  metric 1024  expires 1797sec hoplimit 64
ubuntu@ip-172-31-xx-yy:~$ ping6 -c1 -n www.iana.org
PING www.iana.org(2620:0:2d0:200::8) 56 data bytes
64 bytes from 2620:0:2d0:200::8: icmp_seq=1 ttl=45 time=28.0 ms
```

According to the interfaces(5) manpage, the default behaviour on 14.04 is to **not** accept RAs.  
On 16.04, the default behaviour changed to accept RAs and to enable IPv6 forwarding.  
  
Hope that helps  
Dominik

Answer

**Yes.** That is exactly what I was looking for. Problem is fixed now. **Thank you!**

Answer

I am glad I helped you.  
  
> PS: Out of curiosity, I'll try to look into the 14.04 issue with IPv6 later, using the Oregon Region, too.  
  
I will appreciate that.  
  
Regards.

Answer

Hi fedkad,  
do you have (a) an Egress Only Internet Gateway set up in your VPC, (b) configured it as gateway for ::/0 in the route table and (c) any possible Network ACLs and Security Groups permit for incoming and outgoing IPv6 traffic?  
  
Within your instance, could you check that an IPv6 default route is set as per the "default via … proto ra" line below:

```
dominik@ip-172-aa-bb-cc:~$ ip -6 route show
2001:db8:dead:beef::/64 dev tun0  proto kernel  metric 256  pref medium
2a05:d018:n:n::23 dev eth0  proto kernel  metric 256  pref medium
fe80::/64 dev eth0  proto kernel  metric 256  mtu 9001 pref medium
default via fe80::b8:4aff:feb2:ce81 dev eth0  proto ra  metric 1024  expires 1796sec hoplimit 64 pref medium
```

Best regards  
Dominik

Answer

That was very helpful. Thanks a lot!

EC2 in IPv6 VPC unreachable from outside - why?

Relevant content