How does AWS know whether an EC2 instance rebooted "cleanly"?

0

In the AWS Documentation for EC2 Instance Reboot, it says:

"we perform a hard reboot if the instance does not cleanly shut down within a few minutes."

My question is, how does AWS know whether the EC2 instance shutdown "cleanly"?

For some additional context, we have an EC2 instance running a custom Linux image and we would like to perform a custom "reboot" procedure when the instance is rebooted using the EC2 console/API. So far we have been able to create a custom ACPI Rule that will detect the ACPI power button event that AWS sends and execute our custom procedure, which works. However, about 4 minutes after we click "Reboot instance" from the EC2 Management Console, our instance is getting hard reset, implying that it did not detect that it "cleanly shutdown".

Is there a way to make AWS think that we rebooted cleanly in order to prevent it from issuing a hard reboot after 4 minutes?

MK
asked 11 days ago43 views
1 Answer
0

Hello,

Before answering your questions, let me give a brief about how reboot process works. A controlled reboot is basically a normal shutdown, but instead of turning off the power at the end, the hardware reset procedure is applied so that the firmware takes over and restarts the system, essentially just like when powering up the system.

Moving further On AWS Document it is mentioned : * we perform a hard reboot if the instance does not cleanly shut down within a few minutes which basically means that

1) After you reboot instance via the Amazon EC2 console, a command line tool, or the Amazon EC2 API.

2) Underlying Hardware on which your instance is running will wait few minutes to let the guest OS running inside your instance shutdown properly.

3) If the guest OS took much longer to shutdown due to any reason like user's processes or system services are not shutting down in a timely manner, then Underlying Hardware do "perform a hard reboot" 

Although we might not be able to provide any concrete time frame after which the hard shutdown would be performed if the instance fails to shutdown gracefully/cleanly as it can vary from case to case and we would have to consider the "few minutes" as the reference point. But the 4 minutes which you have mentioned clearly seems to be on a bit higher side resulting in underlying hardware to perform a hard reboot.

In order to achieve your use case what you can do is to use some other alternatives like custom bash script or system manager run command utility to stop the services beforehand which you feel would interrupt the graceful/clean shutdown either by taking too much time to shutdown or due to any other reason and then use the Reboot Instance after that.

Hope it helps!!

Thank you!

answered 8 days ago
  • Thank you for the response. One clarification to my original question: our "custom reboot procedure" actually reboots the OS, but AWS does not seem to think it rebooted, thus it issues a hard reboot a few minutes later (resulting in the image being rebooted twice).

    Example:

    1. Select "Reboot Instance" from EC2 Console, which issues an ACPI "power button" event to the instance.
    2. Our custom Linux image on the instance detects the ACPI event that AWS has sent, which triggers our customized reboot procedure.
    3. From the console of the instance, we can see that the OS was rebooted by our reboot procedure and finishes booting back up within 30 sec or less.
    4. Even though our OS successfully rebooted, AWS must think that we did not actually reboot, so it issues a hard reboot a few minutes later.

    So in regards to the following that you mentioned:

    "2) Underlying Hardware on which your instance is running will wait few minutes to let the guest OS running inside your instance shutdown properly."

    If I am running a customized Linux OS, what is AWS specifically looking for to determine the guest OS instance shutdown properly?

    In other words, how does AWS determine that the OS did not shutdown properly and requires a “hard reboot”?

    Thanks, appreciate your help!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions