- Newest
- Most votes
- Most comments
In this case, here is how I solved the problem:
- Stop the instance.
- Create a new AMI image from the instance.
- Create a new instance from the image.
That's it. The new instance runs fine for many hours now.
So, we can conclude that the issue is not related to any AWS settings (e.g., hardcoded IP, firewalls, routing) and it is not related to applications using too many resources (CPU or RAM).
My best guess is that the issue was probably related to the massive global AWS outage that happened recently.
answered 9 months ago
Based on the information provided, your EC2 instance is likely being stopped by an automated CloudWatch alarm that's monitoring the status checks of your instance. The alarm name "awsec2-i-{ID}-GreaterThanOrEqualToThreshold-StatusCheckFailed" indicates it's monitoring the StatusCheckFailed metric, which triggers when your instance fails its status checks.
According to the AI investigation summary, your instance is failing its status checks after running for about 7 minutes. These status checks verify that your instance is reachable and that the underlying AWS infrastructure is healthy. When an instance fails these checks, it suggests there might be an issue with:
- The operating system on your instance (e.g., running out of memory, CPU, or disk space)
- The underlying hardware or hypervisor
- Network connectivity
- Application issues causing system instability
The "StopInstances" event from "events.amazonaws.com" confirms that AWS CloudWatch Events is automatically stopping your instance in response to the alarm. This is likely a default recovery action configured to stop and restart unhealthy instances.
This is not related to unpaid bills. When AWS suspends accounts for billing issues, they typically don't automatically stop and start instances in this manner.
To troubleshoot this issue:
- Check your instance's system logs for errors or warnings before shutdown
- Monitor CPU, memory, and disk utilization to identify potential resource constraints
- Consider upgrading to a larger instance type if resource constraints are the issue
- Review your application logs for errors that might be causing system instability
- Check if the instance is running out of memory or experiencing kernel panics
- Verify if there are any hardware degradation notifications in your AWS Health Dashboard
If the problem persists, you might need to modify the CloudWatch alarm's behavior or implement a more resilient architecture with multiple instances behind a load balancer.
Sources
Instance or Server keeps shutting down | AWS re:Post
Elevating Your AWS Observability: Unlocking the Power of Amazon CloudWatch Alarms | AWS Cloud Operations Blog
If you are having this issue repeatedly, then it's the instance status check that is failing. Most like reasons are
- you are using an AMI that has issue - maybe you created an AMI with hardcoded IP.
- the EC2 is using all CPU n RAM, change to larger instance type
- Internal firewalls - OS-level firewalls blocking AWS health check agents
- routing issue - misconfigured routes, dropped packets, or disabled network interfaces
Relevant content
asked 4 years ago
