EC2 VM doesn't have Internet connection

0

I have a testing VPC created long time ago, it's connected to our On-Prem network with a VPN connection. We don't assign public IP in EC2 instances in this VPC and only access them through the VPN.

It was working before, but today when I powered up the only EC2 instance in the VPC, I found it lost Internet access. I can still ssh into it with the private IP from On-Prem, but it can't reach any website by curl, I can't ping any public IP including 8.8.8.8 ( DNS resolving is still working ).

[root@ip-10-240-46-19 ec2-user]# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[root@ip-10-240-46-19 ec2-user]# nslookup

www.yahoo.com
Server: 10.240.46.2
Address: 10.240.46.2#53

Non-authoritative answer:
www.yahoo.com canonical name = atsv2-fp-shed.wg1.b.yahoo.com.
Name: atsv2-fp-shed.wg1.b.yahoo.com
Address: 98.137.246.8

[root@ip-10-240-46-19 ec2-user]# ping www.yahoo.com
PING atsv2-fp-shed.wg1.b.yahoo.com (98.137.246.8) 56(84) bytes of data.
^C
--- atsv2-fp-shed.wg1.b.yahoo.com ping statistics ---

[root@ip-10-240-46-19 ec2-user]# yum update
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Could not retrieve mirrorlist http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.list error was
12: Timeout on http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.list: (28, 'Connection timed out after 5000 milliseconds')

[root@ip-10-240-46-19 ec2-user]# curl -v http://amazonlinux.us-west-1.amazonaws.com/2/core/latest/x86_64/mirror.list

  • Trying 52.219.120.48...

  • TCP_NODELAY set

  • Trying 2600:1fa0:c040:250:34db:78a1::...

  • TCP_NODELAY set

tcpdump shows there is no response from public server:
14:36:42.468736 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544269441 ecr 0,nop,wscale 7], length 0
14:36:43.475306 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544270448 ecr 0,nop,wscale 7], length 0
14:36:45.491327 IP 10.240.46.19.42110 > 52.219.24.33.80: Flags [S], seq 3488750344, win 26883, options [mss 8961,sackOK,TS val 1544272464 ecr 0,nop,wscale 7], length 0

I did some troubleshooting:

  1. checked security group:
    There is only one inbound rule to allow TCP 22; one outbound rule to allow "All All 0.0.0.0/0".

  2. Checked VPC routing table, there are three rules:
    10.240.46.0/24 local 0.0.0.0/0 -> internet gateway
    10.0.0.0/8 -> VPN Gateway ( apparent this one still working )

  3. Tried to create a new Internet Gateway and swap it, didn't fix it.

  4. Tried to create a Nat gateway and changed routing 0.0.0.0/0 to use the Nat Gateway, didn't fix it.

  5. removed Nat Gateway, changed it back to use Internet Gateway for 0.0.0.0/0.

  6. Created a new EC2 instance in same subnet with a different private IP, same problem.

Inside the instance, it shows:
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.240.46.1 0.0.0.0 UG 0 0 0 eth0
10.240.46.0 0.0.0.0 255.255.255.224 U 0 0 0 eth0
169.254.169.254 0.0.0.0 255.255.255.255 UH 0 0 0 eth0

Now I run out of idea.
Is there some where else I should be looking?

Thanks,
Jack

gefragt vor 5 Jahren679 Aufrufe
3 Antworten
0

forgot to mention, also checked VPC ACL,its outbound rule has rule 100 to allow any.

beantwortet vor 5 Jahren
0

OK will answer the question myself:

opened support ticket with AWS and the support answered my question:
"In order for your NAT Gateway to work properly, you will have to place it in a public subnet ie. the subnet should have an internet gateway attached to it and you should have subnet specific route tables where the 0.0.0.0/0 on your private subnet is routed to your NAT Gateway and the 0.0.0.0/0 on your public subnet is routed to the Internet Gateway."

https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario2.html

Worked mostly in Azure ( where Internet gateway and NAT gateway were setup directly by Azure Vnet ), I didn't realize how private network should be configured in AWS.

beantwortet vor 4 Jahren
0

Hi jackchen858,

I'm glad that you were able to solve the issue. Please let us know if you need any further help.

Regards,
awstomas

AWS
beantwortet vor 4 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen