How do I troubleshoot high CPU utilization for my Amazon ECS task after I replace my container instances?

Resolution

Note: If you experience high CPU utilization on AWS Fargate, then see How do I troubleshoot high CPU utilization on an Amazon ECS task on Fargate?

View the Amazon ECS service event logs

Check the Amazon ECS service event logs and metrics for CPU-related messages.

View the CPU utilization for the container instance

Complete the following steps:

1. Open the Amazon ECS console.
2. In the navigation pane, choose Clusters, and then select your cluster.
3. Choose the Infrastructure tab.
4. Enter the Amazon Elastic Compute Cloud (Amazon EC2) instance ID in the search bar, and then select your instance
   Note: This action opens the Amazon EC2 console.
5. Review the Resource and Networking tab to check the resource utilization used and available for the container instance.

View the CPU utilization for the Amazon ECS service

Use the Amazon ECS console to check the service health and metrics for the average, minimum, and maximum CPU and memory utilization.

Note: To view container metrics for Amazon ECS tasks, activate Container Insights for your Amazon ECS cluster. For information about available metrics, see Amazon ECS Container Insights metrics.

View your load balancer metrics

Check your load balancer request metrics to determine whether the high CPU utilization occurs when there's a spike in incoming user traffic. For a Network Load Balancer, select the NetworkELB namespace under Metrics, and then choose Per LB to view the ActiveFlowCount and NewFlowCount metrics. If you use an Application Load Balancer, then check the ActiveConnectionCount metric.

Check for encryption overhead

If you activate encryption for Amazon Elastic Block Store (Amazon EBS) volumes, then the encryption and decryption process can cause high CPU utilization.

To check whether high CPU usage occurs because of encryption overhead, compare the CPU and application metrics before and after the EBS volume encryption change.

Make sure that your Amazon ECS tasks have the correct resource allocation

Check your application logs to determine whether your tasks perform CPU-intensive operations. To send your application logs to CloudWatch, see Example Amazon ECS task definition: Route logs to CloudWatch.

Verify that you allocated the correct CPU and memory resources to your Amazon ECS tasks. If CPU utilization is consistently high, then horizontally scale the tasks for the Amazon ECS service.

If horizontal scaling doesn't resolve the issue, then increase the CPU units allocated in the task definition to vertically scale the Amazon ECS service tasks.

Optimize your application code

To analyze and debug distributed applications, use Amazon CloudWatch Application Signals. Trace the path of the request as it travels through the services, functions, and resources in your application. For more information, see Enable your applications on Amazon ECS.