How can I regain access to my EC2 Linux instance ?

Question

I have a RedHat instance I created several years ago as a DNS server, and have been using SSH access from the internet, with a key.
Recently I made what I thought was a trivial change - add "LogLevel VERBOSE" to sshd_config, and I think restarted sshd. Trying to login with SSH now fails. I had also made some provisioning changes earlier, trying to enable IPv6, but the instance had been running normally all the time, for hours after I last touched networking.

I tried to login from the web console. There's a choice of EC2 Instance connect, EC2 Instance connect Endpoint, Session Manager, SSH client, and Serial console.

EC2 Instance Connect previously worked. It now says "EC2 Instance Connect is unable to connect to your instance"
I created an endpoint, but EC2 Instance Connect Endpoint says "Failed to connect to your instance. Error establishing SSH connection to your instance. Try again later." It works on a new instance.

Session Manager said "No IAM". I created an IAM role, which gives a web console on a new instance I made, but on my original instance I get "SSM Agent is not online. The SSM Agent was unable to connect to a Systems Manager endpoint to register itself with the service."

Serial Console says "This instance type is not supported for the EC2 serial console."

I rebooted the instance, but that did not help. The instance is running. The system log and console snapshot show the system running. The DNS server is working normally. nmap from the internet shows:
Not shown: 997 filtered ports
PORT   STATE  SERVICE
22/tcp closed ssh
53/tcp open   domain
80/tcp closed http

If I am completely unable to regain command-line access, is there a way to access the file on disk ?

Accepted Answer

Now fixed.

I created a new instance with default settings which happened to be in the correct availability zone. 
I was then able to stop the original instance and detach the volume, then attach it to the new instance as /dev/sdb. In Linux on the new instance I mounted `/dev/sdb2` on /mnt/sdb2 and edited /mnt/sdb2/etc/ssy/sshd_cconfig.
I then unmounted /mnt/sdb2, detached the volume from the new instance, attached it to the original instance and restarted it.

However, I did not realize that the external IP address had changed, so I still could not login. EC2 connect was not working even though in theory I still had the same keypair, but I was able to repeat the detach/attach/edit process and enable password login. I was then able to login on the internal address from the new instance, and, realizing my mistake, login from the internet using my key.

Update - It occurs to me that I could have made other changes while the volume was mounted on the other instance, such as enabling password logins, or adding other SSH keys to .ssh/authorized_key etc., to let me login from older hosts that might have SSH algorithm incompatibilities.

I still have an issue with keys for EC2 connect for another question.

Answer

Hello.

I suspect that the problem is probably due to a change in the SSH settings.  
Therefore, as described in the document below, stop the EC2 that is currently experiencing the problem, detach the root volume, and then attach it to the normal EC2 and restore the SSH settings to the original.  
After that, if you attach the root volume to the original EC2 and start it, you may be able to connect via SSH.  
https://repost.aws/knowledge-center/ec2-instance-boot-issues

Alternatively, you may want to try changing the EC2 settings so that you can connect via serial console.  
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-serial-console-prerequisites.html

How can I regain access to my EC2 Linux instance ?

Relevant content