CPU usage over 100% on amazon-ssm-agent

0

Hi,
Hi, we have an ec2 instance with amazon-ssm-agent installed on it.. it's topping out with over 100% CPU just a few minutes after restart, causing all other ECS services on the machine to get killed due to lack of CPU.

We've tried two versions - latest = 2.3.701.1
and a downgraded version = 2.3.662.0

Both versions exhibit this behavior...

JonasL
posta 5 anni fa4616 visualizzazioni
4 Risposte
0

Hi JonasL,

Thank you for posting here. Could you please provide the following information to help us investigate this issue?

  1. Instance type
  2. AMI id
  3. Region
  4. Logs about amazon-ssm-agent
con risposta 5 anni fa
0

Sure:
AMI id - amzn-ami-2017.09.f-amazon-ecs-optimized (ami-acb020d5)
Instance type - m5.xlarge
Region - eu-west-1

Logs attached

JonasL
con risposta 5 anni fa
0

Turns out there was a bug in the executable that the ssm-agent tried to restart. This caused the ssm-agent to re-try to execute the script even though it was failing - causing the 100% cpu.

JonasL
con risposta 5 anni fa
0

Hi JonasL,
Base on the log, there are over 180 command executions within 2 mintues. All the commands have the same content, all excutions completed with failed status (no retry).

The high CPU usage was related to the nubmer of the commands have been executed.

Do you have configuration (StateManager, CloudWatch) that sends commands to the instance after restart?

Regards,
Shihua

con risposta 5 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande