Windows server instance ec2 restarted suddenly with extensive loss of information and work

Question

Our Windows server instance i-0bb861ebbbcf0585a restarted suddenly (against all expectations of AWS services and failure control). Lost of ongoing unsaved work. Checking system later for logs later for anything in the system being the cause does not indicate any obvious cause.

1. Machine is RDP accessible only from a specific windows machine.
2. Cloudtrail log seems to indicate an aws IP address and shows the standard UpdateInstanceInformation which is not informative by itself and the JSON also is  uninformative showing 'eventType' as "AWSAPICall". 
3. A subsequent event was 'RegisterManageInstance' - not sure if this is informative by itself to get help.

Can an AWS engineer answer? It is a serious matter for all of us relying on AWS failsafe etc.

Thanks.

Answer

Hi, sg_rsx

Is it possible that this instance got terminated, then got rebuilt by an IaC configuration (CloudFormation, Terraform, etc) and/or CM tool (Ansible, Salt, etc)?  Stopping an instance can do this, if it was built using IaC (rebooting typically doesn't trigger IaC termination & rebuild).  Just in case, here's information on how to suppress termination for an EC2 instance, if this is applicable at all.  The UpdateInstanceInformation & RegisterManagedInstance made me think of this.

`Select your EC2 Instance > Action > Instance Settings > Change termination protection > Check or uncheck the box as needed. You can disable this protection at any time through the Instance Settings.`

One other possibility that comes to mind is if it's an EOL instance that got updated - this happens rarely, but I've experienced it.  AWS usually notifies the customer of this well in advance, so not as likely of a cause.

I'm hoping there's either an EBS snapshot or EBS volume (with default termination protection enabled) you could recover the data from, by restoring/reattaching to the Windows server instance.

Here's how you can open a case with AWS Enterprise Support, from within the AWS account which controls the Windows server instance.

**To contact AWS Support**
* Sign in and navigate to the AWS Support Center (you can find this by typing 'support center' in the search bar at top)
* Choose Create case
* On the Create case page, select Account and billing and fill in the required fields on the form
* Choose your response preference: **Web:** You'll receive an email response from an AWS Support representative  **Chat:** quick, instant chat option with Support Engineer (for Business & Enterprise customers).

If this helps answer your question, please choose this as the Accepted Answer so others on re:Post may benefit - Thank you

Answer

If you are on a Developer support plan or above then log a support call with AWS to try and get a root cause of for the loss of service, and of you're on one of the Enterprise support plans you may want to engage an account manger.

AWS doesn't guarantee 100% uptime for the EC2 service, but there is an SLA where, if an availability %age is not met, you may be eligible for service credits https://aws.amazon.com/compute/sla/

Windows server instance ec2 restarted suddenly with extensive loss of information and work

Relevant content