EC2 mysteriously loses connectivity - telnet google.com 80 not working - AMI on another EC2 works without problems

Question

I have an ec2 instance on a public subnet with Ubuntu running for months without problems. Today, when connecting to it via ssh I have seen the following error:

```
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings
```

Investigating a little more in depth I see that a simple

```
telnet google.com 80
Trying 172.217.19.238...
```

does not work, it does not establish a connection. 
I have also tried

```
nslookup google.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	google.com
Address: 172.217.19.238
Name:	google.com
Address: 2a00:1450:4007:80d::200e
```

and it works fine. A telnet to another instance of the same vpc and subnet works ok. 
The systemd-resolved.service is up and without errors:

```
systemctl status systemd-resolved.service
● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/lib/systemd/system/systemd-resolved.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2022-08-23 10:37:22 UTC; 46min ago
       Docs: man:systemd-resolved.service(8)
             https://www.freedesktop.org/wiki/Software/systemd/resolved
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
   Main PID: 1586 (systemd-resolve)
     Status: "Processing requests..."
      Tasks: 1 (limit: 4637)
     Memory: 4.3M
     CGroup: /system.slice/systemd-resolved.service
             └─1586 /lib/systemd/systemd-resolved

Aug 23 10:37:22 ip-172-31-34-169 systemd[1]: Starting Network Name Resolution...
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Positive Trust Anchors:
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237>
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr>
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Using system hostname 'ip-172-31-34-169'.
Aug 23 10:37:22 ip-172-31-34-169 systemd[1]: Started Network Name Resolution.
```

I have created an AMI of this instance and I have raised another ec2 with this AMI, and everything works correctly, the new ec2 is in the same vpc and subnet and has the same security group, so I rule out connectivity problems in the vpc, route table , ACL, internet gateway etc...

Could it be due to some problem in the network interface?
Any idea what could be happening?

Answer

I suspect issue with the Network interface itself. You probably can try enabling VPC flow logs and check the logs to understand where traffic is getting dropped.

Answer

Try creating new ENI in same subnet and AZ and then connect using this new ENI.If it works then definitely issue with your primary ENI. Probably, you can create image out of your EC2 and create new EC2 using that image. I hope this helps.

EC2 mysteriously loses connectivity - telnet google.com 80 not working - AMI on another EC2 works without problems

Relevant content