How do I troubleshoot scaling issues with my Amazon EC2 Auto Scaling group?

5 minute read

My Amazon EC2 Auto Scaling group doesn't scale correctly.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Your Amazon EC2 Auto Scaling group doesn't follow the scaling policy

Check your scaling policies to see whether an event invokes more than one policy. If two policies scale at the same time, then Amazon EC2 Auto Scaling follows the policy with the greater effect. For example, you have one policy to add two Amazon Elastic Compute Cloud (Amazon EC2) instances and another policy to add four instances. When both policies scale at the same time, EC2 Auto Scaling adds four instances.

If you have multiple target tracking scaling policies, then only one of the policies scales out. However, all target tracking scaling policies must be ready to scale in at the same time for Amazon EC2 Auto Scaling to scale-in action.

Your Amazon EC2 Auto Scaling group is scaling out instead of scaling in

Check whether your scale-out policy and scale-in policy scale at the same time. When both policies scale at the same time, Amazon EC2 Auto Scaling follows the scale-out policy to confirm availability.

To check whether scale-out and scale-in policies scale at the same time, take the following actions:

Review your alarm history on the Amazon CloudWatch console to check if Amazon CloudWatch invokes the associated alarms at the same time.
Review your group's activity history on the Amazon EC2 console to check whether the associated scheduled actions run at the same time.

Your Amazon EC2 Auto Scaling group doesn't respond to your CloudWatch alarm or health check replacement

To determine why your group doesn't respond to your CloudWatch alarm or health check replacement, take the following actions:

Check whether your group already reached its minimum or maximum number of instances. To find the instance capacity, review your group's details on the Amazon EC2 console. When your group reaches capacity, Amazon EC2 Auto Scaling doesn't show an error message in the group's activity history.
Check suspended processes for your group. Amazon EC2 Auto Scaling doesn't evaluate a CloudWatch scaling policy for suspended Terminate,Launch, or AlarmNotification processes. Amazon EC2 Auto Scaling also doesn't evaluate a health check replacement for suspended ReplaceUnhealthy or HealthCheck processes.
Check whether you turned on scale-in protection in any of the instances in your group. When you turn on scale-in protection for an instance, Amazon EC2 Auto Scaling doesn't use instance to scale down your group. You must turn off scale-in protection for the group to scale down.
If you set up a simple scaling policy, then check whether your instances are in a cooldown period. Simple scaling policies initiate activities only after the cooldown period ends.
If your target tracking scaling policy doesn't scale down your group, then check whether the scale-in portion is deactivated in the policy.
Check whether you deactivated your scaling policy. When you turn off a scaling policy, the group doesn't change.
Check whether you configured the step adjustment for your step scaling policy to react to the size of the alarm breach.
If you set up a step scaling policy or target tracking scaling policy, then check for an in-progress instance warmup. Amazon EC2 Auto Scaling counts instances toward the group metrics only after the warmup period ends.
Confirm that your CloudWatch alarm correctly invokes scaling activity.
If you configured a lifecycle hook, then either complete the lifecycle hook or wait for the timeout period to end.

Your instance is stuck in the Pending:Wait or Terminating:Wait state during scaling activity

If you configured a lifecycle hook for your group, then an instance might be in the Pending:Wait or Terminating:Wait state. Scaling activities that start because of simple scaling policies are paused until the instance leaves the Pending:Wait or Terminating:Wait state.

To determine whether you configured a lifecycle hook, run the describe-lifecycle-hooks command:

aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name my-asg-name

To continue the scaling process, wait for the timeout period to end. Then, complete the lifecycle hook.

Note: By default, the timeout period is 1 hour.

For information on lifecycle hook management in the Amazon EC2 console, see Amazon EC2 Auto Scaling lifecycle hooks.

Your Amazon EC2 Auto Scaling group doesn't respond to a scheduled action

If your group isn't responding to a scheduled scaling action, then take the following actions:

Check whether you configured a time zone for the scheduled action. If you didn't set a time zone, then create a scheduled action.
Note: Scheduled actions are set in Coordinated Universal Time (UTC) by default. If you set a time zone, then verify that the action runs based in that time zone.
Review your group's activity history on the Amazon EC2 console to determine whether scaling activity conflicts with your scheduled action.
If you schedule scale-out and scale-in actions, then check that you scheduled one action for scaling out and another action for scaling in. You can't use the same scheduled action to scale in and scale out.

You reached your EC2 instance quota

If your group isn't scaling because you reached your EC2 instance quota, then you receive a message that resembles the following:

"Launching a new EC2 instance. Status Reason: Your quota allows for 0 more running instance(s). You requested at least 1. Launching EC2 instance failed."

To increase the quota, contact AWS Support. For information about quotas, see AWS service quotas.