I have an ec2 instance on a public subnet with Ubuntu running for months without problems. Today, when connecting to it via ssh I have seen the following error:
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings
Investigating a little more in depth I see that a simple
telnet google.com 80
Trying 172.217.19.238...
does not work, it does not establish a connection.
I have also tried
nslookup google.com
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: google.com
Address: 172.217.19.238
Name: google.com
Address: 2a00:1450:4007:80d::200e
and it works fine. A telnet to another instance of the same vpc and subnet works ok.
The systemd-resolved.service is up and without errors:
systemctl status systemd-resolved.service
● systemd-resolved.service - Network Name Resolution
Loaded: loaded (/lib/systemd/system/systemd-resolved.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2022-08-23 10:37:22 UTC; 46min ago
Docs: man:systemd-resolved.service(8)
https://www.freedesktop.org/wiki/Software/systemd/resolved
https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
Main PID: 1586 (systemd-resolve)
Status: "Processing requests..."
Tasks: 1 (limit: 4637)
Memory: 4.3M
CGroup: /system.slice/systemd-resolved.service
└─1586 /lib/systemd/systemd-resolved
Aug 23 10:37:22 ip-172-31-34-169 systemd[1]: Starting Network Name Resolution...
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Positive Trust Anchors:
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237>
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr>
Aug 23 10:37:22 ip-172-31-34-169 systemd-resolved[1586]: Using system hostname 'ip-172-31-34-169'.
Aug 23 10:37:22 ip-172-31-34-169 systemd[1]: Started Network Name Resolution.
I have created an AMI of this instance and I have raised another ec2 with this AMI, and everything works correctly, the new ec2 is in the same vpc and subnet and has the same security group, so I rule out connectivity problems in the vpc, route table , ACL, internet gateway etc...
Could it be due to some problem in the network interface?
Any idea what could be happening?
I have put a flow log to the ENI but the only information it gives me is that the traffic has been rejected: ${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} 2 XXXXXXXX eni-09b00dc4143592a34 172.31.34.169 172.217.18.206 34714 80 6 1 60 1661329646 1661329677 REJECT OK