By using AWS re:Post, you agree to the Terms of Use
/SSH broken as of 1st March/

SSH broken as of 1st March

0

We have an automated codepipeline that builds packer images based on amazon linux AMI updates. This pipeline has worked flawlessly for several years. Yesterday however it broke with : ==> amazon-ebs: Waiting for instance (xxxxxxx) to become ready... ==> amazon-ebs: Using ssh communicator to connect: xx.xx.xx.xx ==> amazon-ebs: Waiting for SSH to become available... ==> amazon-ebs: Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods publickey none, no supported methods remain

We have changed nothing , something in the base AWS Li2 image has broken this we think.

  • Interestingly if I run the image created on a T2 micro instead of a T3 micro , I can ssh to it.

  • Please don't post any personal information in the question. If you would like to share personal information, create a support case instead.

2 Answers
0

Hello,

Thank you for asking your question in AWS re:Post. I will recommend to create a Support Case so we can investigate further.

Amazon Linux 2 Public AMIs will have necessary pre-requisites for running t3.micro or any Nitro system instances. You may also investigate the CloudTrail logs for CodePipeline or any EC2 API calls for the EC2 instance.

SUPPORT ENGINEER
answered 3 months ago
0

I raised it with support at the same time as posting on here. They came back yesterday to say there is a problem with the latest AMI , which breaks ssh on T3 , and for anyone interested here's the workaround:-

                                            ****************************************

Creating derived AMIs (by creating AMIs from EBS snapshots of EC2 instances) and not cleaning up cloud-init cached state may experience issues with cloud-init executing in AMIs derived from the amzn2-ami-hvm-2.0.20220218.0 Amazon Linux 2 AMI (released on 2022-02-28) .

We recommend the following workaround currently, until the issue is fixed:

Remove cloud-init's cached state using cloud-init clean.

Create an empty file at /etc/systemd/system/cloud-init-local.service.d/skip.conf before creating an EBS snapshot/AMI.

This file will allow AMI's derived from amzn2-ami-hvm-2.0.20220218.0 Amazon Linux 2 AMI to not run into this issue.

Example commands to run before snapshotting an AMI:

cloud-init clean mkdir -p /etc/systemd/system/cloud-init-local.service.d touch /etc/systemd/system/cloud-init-local.service.d/skip.conf

No other versions of the Amazon Linux 2 AMI contains this issue hence, you may also consider rolling back to previous AMI version if the above shared workaround is not feasible to you.

                                            *************************************************

We sincerely apologize for any inconvenience this may have caused, and regret that it has affected your production.

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions