EC2 snapshot is corrupted and the EC2 instance has experienced a crash.

Question

We attempted a straightforward process of terminating and restarting our EC2 instance, but were met with a critical issue. To remedy the situation, we tried implementing a snapshot for recovery. However, during this process, we encountered an error indicating the file was empty, which led all our recovery efforts to be in vain.

Before we stopped the EC2 instance, the volume was completely filled and then emptied. We then proceeded with the shutdown, but have been unable to restart it since. We strongly suspect this may have been the root cause of the issue we're experiencing.

We would greatly appreciate your guidance on how we can resolve this problem. We seek a solution as quickly as possible due to the impact it is having on our business operations.

Answer

From your question it sounds like you removed the contents of the main drive before taking the snapshot.  If so the only option, I can see is using a previous backup and restoring from that.

As it is mission critical, I would suggest that you implement the following:
* Ensure that you have recurring backups
* Enable auto scaling so that the system can recover if the instance stops working.
* Ensure you have Business support at a minimum.

If you already have Business support, I would reach out to them to get this resolved as soon as possible.
- https://aws.amazon.com/premiumsupport/plans/business/

Hope this helps.

EC2 snapshot is corrupted and the EC2 instance has experienced a crash.

Relevant content