Unable to connect (101: Network is unreachable)
A few points:
- I had set this server up on 22nd August
- Every 26 days I encounter the following issue and to date, I have set 3-4 servers in different accounts but still face the same issue
- I sense it has something to do with my security group but I have enabled the ones which are required and nothing more
Error Err:1 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu jammy InRelease Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (13.233.101.120), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (3.110.169.167), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (3.110.200.74), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (13.126.16.60), connection timed out Err:2 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu jammy-updates InRelease Unable to connect to ap-south-1.ec2.archive.ubuntu.com:http: Err:3 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu jammy-backports InRelease Unable to connect to ap-south-1.ec2.archive.ubuntu.com:http: Err:4 http://security.ubuntu.com/ubuntu jammy-security InRelease Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.81), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out Reading package lists... Done Building dependency tree... Done Reading state information... Done 16 packages can be upgraded. Run 'apt list --upgradable' to see them. W: Failed to fetch http://ap-south-1.ec2.archive.ubuntu.com/ubuntu/dists/jammy/InRelease Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (13.233.101.120), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (3.110.169.167), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (3.110.200.74), connection timed out Could not connect to ap-south-1.ec2.archive.ubuntu.com:80 (13.126.16.60), connection timed out W: Failed to fetch http://ap-south-1.ec2.archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease Unable to connect to ap-south-1.ec2.archive.ubuntu.com:http: W: Failed to fetch http://ap-south-1.ec2.archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease Unable to connect to ap-south-1.ec2.archive.ubuntu.com:http: W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/jammy-security/InRelease Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::102). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::103). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::16). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4000:1::19). - connect (101: Network is unreachable) Cannot initiate the connection to security.ubuntu.com:80 (2620:2d:4002:1::101). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.39), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.81), connection timed out Could not connect to security.ubuntu.com:80 (185.125.190.36), connection timed out W: Some index files failed to download. They have been ignored, or old ones used instead.
- Newest
- Most votes
- Most comments
I am assuming you have verified the server portion of your application is up and running on the server. The issue may be a change in IP address. When you create an EC2 instance and let the creation process assign a public IP address - this address is not permanent. When your EC2 server reboots, it will pull a new public IP. I am guessing you may be rebooting the server every 26 days and this is happening. To ensure you keep the same public IP address between reboots, you need to allocate an elastic IP address manually and assign it to your EC2 instance.
If this is not the answer, can you describe what you do to restore service after the 26 day problem occurs?
The Ubuntu OS is trying to reach ap-south-1.ec2.archive.ubuntu.com and security.ubuntu.com, both on port 80, but is not able to establish the connection.
That single rule in the outbound security group rule should be anough to allow this, unless there is also a NACL in place that would restrict this (by default there isn't, it would have to be something that was added) but it's still worth checking there isn't a NACL in place for the subnet that the EC2 is in https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html
Assuming the SG is alright and there is no NACL blocking things, how do you connect onto the EC2? Do you use SSH over the internet, because if you do then that means the associated infrastructure (internet gateway, route tabe, etc.) is likely all good. If it's another method, e.g. over Direct Connect or VPN, or via a bastion, then can you describe that.
From the host are you able to connect to other sites on the intenet, e.g. curl www.google.com ?
I am able to ping.
Relevant content
- asked 18 days ago
- asked a month ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago
No, I did not restart until I started seeing these issues. I have allocated the Elastic IP as well that you have mentioned but I still have the same issue.
I am not able to restore services unless I recreate another instance and redeploy my applications there
Are you able to verify all ports are up, service running and responding locally on the server... or from a server in the same subnet? Taking some more guesses that perhaps an attached drive has filled and is preventing a service from starting as it might not be able to write logs or such??
Nop. No memory issue. I am now facing this issue on a newly setup server.
I reread your question and wanted to take a different route. Based on the output you provided, it looks like you have a Ubuntu Jammy EC2 instance and running an apt update (or such). It is failing as it can't reach the various package managers. So a few more questions: