AWS EC2 - after restoring a RHEL 7 EC2 from a backup, the ssh host keys change
I ran a test whereby I restored a RHEL 7 EC2 instance. After it started, I was able to ssh into the instance using the same private key that I use for the instance that was backed up. However I noticed that the /etc/ssh/ssh_ key files had all changed, and an application, Oracle's secure global desktop, could not longer connect, due to the changed host keys.
What would have caused the host keys to change?
Is it something that AWS restore does?
Is it something that sshd does on startup? E.g. it detects that the hostname changed (AWS assigns a new hostname to the instance) and recreates the host keys as a result?
Is it something that RHEL 7 does?
I suppose I could just backup the /etc/ssh directory and restore the files after a restore, but I would like to think that I can use AWS backup / restore out of the box to preserve the configuration of an instance during the restore operation.
There is a very old AWS forums post:
that refers to this problem, but the post says that the issue was resolved in the AWS linux AMI.
Thanks for the help.
The host key may have been regenerated by cloud-init. cloud-init will compare the running instance id to that it cached previously (i.e. the instance id the backup was taken from) and find they're different, so will run the first-boot modules again, which includes the ssh host key.
Have a look at
/var/log/cloud-init.log to see if it mentions there it's doing anything with the host key?
You can provide the host keys which should be installed, or at least prevent it from deleting the old ones, via a configuration file you can pass in user data (see last link above for docs).
That was what created the host keys, thanks.
I read the doc you recommend, but don't see how to prevent the deleting of the host keys. Can you provide some more details? When running AWS Restore, AWS will launch an EC2 instance, so I don't see how to pass anything to cloud init?
Is this using the restore feature in AWS Backup? Looks like you should be able to provide UserData under the "Advanced" section in the UI, or in the UserData field if you're doing it from the CLI etc: https://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-ec2.html, but I haven't tested that myself.
AWS EC2 - after restoring a RHEL 7 EC2 from a backup, the ssh host keys changeAccepted Answerasked 5 months ago
change EC2 parameters (cores, RAM) after I have installed a GUI and some softwaresasked 5 months ago
SSH into EC2 stops accepting connections after about 8 minutesasked 5 months ago
EC2 instance inaccessible after RDP changeasked 2 years ago
What is the difference between EC2 Instance Connect and Session Manager SSH connections?Accepted Answerasked 3 years ago
Unable to access ec2 instance | SSH Connection Timed Outasked 10 months ago
Unable to SSH into instanceasked 2 years ago
Unable to SSH into my EC2 Instance Permission denied (publickey)asked 3 years ago
Cannot SSH into any EC2s a few minutes after startingasked 2 years ago
A new EC2 instance is created after termination of the only instanceAccepted Answerasked 2 years ago