Can't access internet from EC2 - security group, ACL and route tabl checked

Question

Hello, I can't manage to get my EC2s to access internet.  
  
* they both have a EIP attached;  
* they both have a security group allowing all TCP traffic inbound and outbound from any IP;  
* they both are in a subnet in a VPC with an internet gateway attached to it;  
* the Network ACL attached to the their subnet accept all traffic via any protocol on any port from any IP;  
* the route table associated to their subnet redirect all traffic to the internet gateway  
  
I should say that I'm creating the stack via cloud formation; I did a similar setup via the console and it worked fine but i can't figure out what I missed via cloud formation. If it's any help i'll provide link to the template JSON and Designer screenshot below.

Accepted Answer

(see previous post on not relying on using ping as that is blocked by your egress security rules)  
  
Yes. Having no public ip address is normal. Traffic from the public ip address will get routed to your instance, but your instance will only have a private ip address.

Answer

Hello  
the cloudformation template on the link, does not have an association between the EC2 instance and the EIP.  At least, I couldnt see it.  Maybe that's the cause of the problem.  
once you ran your cloudformation template, it would be easy to go back to the console and check the newly created VPC and all the  other components.  
  
here is a link to a sample template that has EIP and EC2

hope this helps,  
RT

Answer

That was it... I thought i tried to allow all traffic (not only tcp) at some point; I clearly was wrong. Thank you very much for your time and help.

Answer

So i managed to create the exact same setup via the console and it works fine; I have access to internet.   
  
So i opened two browser window and check every resources; I started from my EC2; clicking on every resources linked to it in the description tab and oing so for every resources so i check everything possible. After i did this and compare my two setup it appears they are the same.   
  
So i checked the '/var/log/cloud-init-output.log' on both EC2; the net info are pretty similar. But I may have detected a problem in the route info;   
The EC2 working have following routes (Destination - Gateway):  
 * 0.0.0.0 - 10.0.0.1  
 * 10.0.0.0 - 0.0.0.0  
 * 10.0.0.1 - 0.0.0.0  
  
The EC2 **not** working have following routes (Destination - Gateway):  
 ** 0.0.0.0 - 10.0.**1*.1  
 ** 10.0.**1*.0 - 0.0.0.0  
 ** 10.0.**1*.1 - 0.0.0.0  
  
--I don't think those **1** should be there; it does not look like how my route table are defined in the console.--  
So it was just my subnet IP that was set up like this in cloud formation; so as of now i'm definitely stucked; i don't know what i may have missed. If one of you could import the stack via cloudformation and check the resulting resources it would be greatly appreciated; you need to be in N.Virginia region and have a key pair available.  
  
Edited by: OURSONDECHINE on Jun 11, 2019 5:22 AM

Answer

Hi. You say you can connect to your instance via SSH. So what exactly is your issue? Perhaps if you provide some examples of what is not working and possibly what is working we might be able to narrow down the issue?  
  
Matt.

Answer

Ping will not work as you do not allow ICMP packets through your security group egress rules. So you cannot rely on ping to determine if your instance has access to internet or not.  
  
Maybe try using curl or something that requires TCP to do your tests?

Answer

Yes, as you guessed my user data does not work. I had no idea why at first the i did a ping on amazon.com to check i had internet access and I did not.  
  
Thanks for the log path; I just checked it and all I have in here is infos about my 'Net device info' and 'Route IPv4 info'. What seem maybe odd is that I have my localhost address (127.0.0.1); my local network address (10.0.**.**) but i don't see any public address in here. Is that normal ?  
  
--For the others you mentioned; i did check them again and everything seems fine.--   
--It seems that two routes tables are being created; one attach to the subnet another one is the VPC main route table; the second one has no traffic redirected to the IG; I guess here is my problem. I'm gonna investigate in this.--  
So from here   it seems my current setup is correct; I'm lost  
  
I also want to point out that I have 2 EC2 both with an EIP each and they are in the exact same state but have different userdata.  
  
Edited by: OURSONDECHINE on Jun 11, 2019 1:05 AM  
  
Edited by: OURSONDECHINE on Jun 11, 2019 1:41 AM  
  
Edited by: OURSONDECHINE on Jun 11, 2019 1:46 AM

Answer

Hello  
sorry for missing that line.  Since you can connect to it via SSH and your security group allows all TCP ports, lets assume that part is fine.  
so could it be something with your instance?  
  
I saw your userdata and it downloads content from the internet.  Is that part not working?  
you could check the logs for info regarding this.

```
 /var/log/cloud-init-output.log
```

when you connect to the instance, can you do a nslookup to check the ipaddress or a domain? that's to test if it can go out as ping (ICMP) and traceroute (UDP) will not work to check because of the security group.   
  
I would check via the console:  
- security group and its association with the vpc, nacl and its association with subnets,  routetable and its association with the subnets,  and then the instance logs  
  
hope this gives you some ideas, sorry cant be of more help  
RT

Answer

Hey, thank you for taking the time to check on my problem. Actually I do link the EIPs to the EC2s on the EIP allocations via the 'InstanceId' property (line 87 and 95 in the template).  
  
As described in here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-eip.html#cfn-ec2-eip-instanceid  
  
Also I can cnonnect to the EC2 via ssh on said IP so i guess the problem is not there.  
  
I did went back to the console to check everything but I can't manage to find what's missing :/  
  
Edited by: OURSONDECHINE on Jun 10, 2019 7:27 AM

Answer

Hey ! Thanks for taking some time to try and resolve my problem. As you can see in another answer below, I first tried some pat-get install in my user data but i had nothing installed after the stack was fully complete. So I tried to ping amazon.com and it appears it does not work.

Answer

I'm not 100% sure of my user data yet; I just connected to th EC2, see no soft installed, tried to ping, figured it was the problem. But the fact that i didn't allow ICMP was the problem.  
  
I'm now gonna debug my user data and continue with my setup. But thank you very much for your time and help !

Answer

Hello  
I tried to download the files you have on your userdata

```
https://download.mono-project.com/repo/ubuntu stable-bionic main -->file not found
```

It should start with something like

```
https://download.mono-project.com/repo/ubuntu/dists/stable-bionic/main/
I say start because the path is a bit longer as you need to choose your platform
```

for the route table, you need two entries the local network

```
Destination= YOUR internal IPs
Target= local
Destination= 0.0.0.0/0
Target= IGW-id
```

The IGW will stamp the EIP address when going out to the internet from your instance and do the NAT

if your subnet ip address range is not in the route table, you will have to add it.  If your local IP address for your instance is different from what is on the route table, review what it should be and update the template.  
  
when you SSH to your instance try to do a

```
wget http://www.amazon.com
```

just to see if you can really go out to the internet  
  
hope this helps  
RT

Can't access internet from EC2 - security group, ACL and route tabl checked

Relevant content