AWS EC2 - after restoring a RHEL 7 EC2 from a backup, the ssh host keys change

0

I ran a test whereby I restored a RHEL 7 EC2 instance. After it started, I was able to ssh into the instance using the same private key that I use for the instance that was backed up. However I noticed that the /etc/ssh/ssh_ key files had all changed, and an application, Oracle's secure global desktop, could not longer connect, due to the changed host keys.

What would have caused the host keys to change?

Is it something that AWS restore does?

Is it something that sshd does on startup? E.g. it detects that the hostname changed (AWS assigns a new hostname to the instance) and recreates the host keys as a result?

Is it something that RHEL 7 does?

I suppose I could just backup the /etc/ssh directory and restore the files after a restore, but I would like to think that I can use AWS backup / restore out of the box to preserve the configuration of an instance during the restore operation.

There is a very old AWS forums post:

https://forums.aws.amazon.com/thread.jspa?threadID=40450

that refers to this problem, but the post says that the issue was resolved in the AWS linux AMI.

Thanks for the help.

asked 2 years ago757 views
1 Answer
0
Accepted Answer

The host key may have been regenerated by cloud-init. cloud-init will compare the running instance id to that it cached previously (i.e. the instance id the backup was taken from) and find they're different, so will run the first-boot modules again, which includes the ssh host key.

Have a look at /var/log/cloud-init.log to see if it mentions there it's doing anything with the host key?

You can provide the host keys which should be installed, or at least prevent it from deleting the old ones, via a configuration file you can pass in user data (see last link above for docs).

profile pictureAWS
EXPERT
James_S
answered 2 years ago
  • That was what created the host keys, thanks.

    I read the doc you recommend, but don't see how to prevent the deleting of the host keys. Can you provide some more details? When running AWS Restore, AWS will launch an EC2 instance, so I don't see how to pass anything to cloud init?

  • Is this using the restore feature in AWS Backup? Looks like you should be able to provide UserData under the "Advanced" section in the UI, or in the UserData field if you're doing it from the CLI etc: https://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-ec2.html, but I haven't tested that myself.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions