EC2 instance 100% CPU utilisation then stuck at 10%, Completely frozen Can't even ssh in

1

An ec2 t2.micro instance (linux) shows a very abnormal behavior. I have a very light application which usually consumes less than 3% of CPU usage. Suddenly (almost each 4th day) the instance jumps at 100% CPU usage for around 12 hours and then stucks to 10% .

When the instance is stuck, I can't even ssh in, the ssh connection just times out. Ine the image below the behavior is shown. After I stop & start the instance again, everything works again.

Some other details:

  • the instance has a security group that only allows my ip address to connect via ssh.
  • The app runs in a docker container and it barely receives traffic.

Any ideas would be highly appreciated :)

https://imgur.com/a/j5BOf61

1 Antwort
2

This can easily be caused by some sort of (default) scheduled activity on the instance. Unfortunately, you have not disclosed the OS type / Linux distribution.

You may consider upscaling the instance and/or enabling T2/T3 Unlimited mode in order to accommodate the workload without causing a disruption.

You may also consider enabling T2/T3 Unlimited mode just so you will be able to connect to the instance when the activity is happening, and then investigating the cause.

See also https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-unlimited-mode.html

SMP
beantwortet vor 2 Jahren
  • It's a linux instance (AMI: 099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20211129)

    What kind of default scheduled activity can be running on the instance?

  • You can find the scheduled jobs by running commands such as

    find /etc/cron.* -type f -not -name .placeholder

    systemctl --type=timer

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen