CPU usage over 100% on amazon-ssm-agent

0

Hi,
Hi, we have an ec2 instance with amazon-ssm-agent installed on it.. it's topping out with over 100% CPU just a few minutes after restart, causing all other ECS services on the machine to get killed due to lack of CPU.

We've tried two versions - latest = 2.3.701.1
and a downgraded version = 2.3.662.0

Both versions exhibit this behavior...

JonasL
demandé il y a 5 ans4615 vues
4 réponses
0

Hi JonasL,

Thank you for posting here. Could you please provide the following information to help us investigate this issue?

  1. Instance type
  2. AMI id
  3. Region
  4. Logs about amazon-ssm-agent
répondu il y a 5 ans
0

Sure:
AMI id - amzn-ami-2017.09.f-amazon-ecs-optimized (ami-acb020d5)
Instance type - m5.xlarge
Region - eu-west-1

Logs attached

JonasL
répondu il y a 5 ans
0

Turns out there was a bug in the executable that the ssm-agent tried to restart. This caused the ssm-agent to re-try to execute the script even though it was failing - causing the 100% cpu.

JonasL
répondu il y a 5 ans
0

Hi JonasL,
Base on the log, there are over 180 command executions within 2 mintues. All the commands have the same content, all excutions completed with failed status (no retry).

The high CPU usage was related to the nubmer of the commands have been executed.

Do you have configuration (StateManager, CloudWatch) that sends commands to the instance after restart?

Regards,
Shihua

répondu il y a 5 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions