EC2 instance 100% CPU utilisation then stuck at 10%, Completely frozen Can't even ssh in

1

An ec2 t2.micro instance (linux) shows a very abnormal behavior. I have a very light application which usually consumes less than 3% of CPU usage. Suddenly (almost each 4th day) the instance jumps at 100% CPU usage for around 12 hours and then stucks to 10% .

When the instance is stuck, I can't even ssh in, the ssh connection just times out. Ine the image below the behavior is shown. After I stop & start the instance again, everything works again.

Some other details:

  • the instance has a security group that only allows my ip address to connect via ssh.
  • The app runs in a docker container and it barely receives traffic.

Any ideas would be highly appreciated :)

https://imgur.com/a/j5BOf61

1 Answer
2

This can easily be caused by some sort of (default) scheduled activity on the instance. Unfortunately, you have not disclosed the OS type / Linux distribution.

You may consider upscaling the instance and/or enabling T2/T3 Unlimited mode in order to accommodate the workload without causing a disruption.

You may also consider enabling T2/T3 Unlimited mode just so you will be able to connect to the instance when the activity is happening, and then investigating the cause.

See also https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-unlimited-mode.html

SMP
answered 2 years ago
  • It's a linux instance (AMI: 099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20211129)

    What kind of default scheduled activity can be running on the instance?

  • You can find the scheduled jobs by running commands such as

    find /etc/cron.* -type f -not -name .placeholder

    systemctl --type=timer

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions