High CPU Usage and Celery Malfunction Issue on ECS with Celery + SQS

Increased Task Activity: Sometimes, during deployments, ECS might briefly spin up additional tasks, which can temporarily increase CPU utilization. If your deployment isn’t completing properly or tasks are lingering, the CPU usage could stay high. You can check the deployment status in ECS and monitor CPU utilization via CloudWatch to see if this is happening ( Bobcares ).

Celery Task Behavior: Celery can spike CPU usage if tasks are queued inappropriately or if long-running tasks are being retried frequently. Check your Celery worker logs for signs of failed or stuck tasks, as they might be consuming excessive CPU. Also, ensure that task prefetching is configured properly—an incorrect configuration can lead to the workers fetching more tasks than they can handle, leading to CPU spikes​( Bobcares ).

Changes in SQS: If there has been a sudden increase in the number of messages in your SQS queue or changes in how messages are delivered, it could lead to higher CPU usage as Celery workers handle more tasks than usual. You should monitor the SQS queue for sudden surges in traffic​( GitHub ).

Django or Library Updates: Check for recent updates or changes in Django, Celery, or any related Python libraries. Sometimes, a new version of a library might introduce inefficiencies that weren’t present in previous versions.

Here are some steps to troubleshoot further:

Logs: Check both your ECS task logs and Celery worker logs for any anomalies or errors that might explain the spike. CloudWatch Metrics: Review the ECS task CPU and memory metrics in CloudWatch to see if there's a pattern or if the spike coincides with specific events. Task Scaling: If your service is handling more traffic, consider scaling up your ECS tasks or increasing the task CPU and memory allocation​( Bobcares ).