Why doesn't my CloudWatch alarm's state change invoke my Amazon EC2 Auto Scaling policy?

6 minute read
0

My Amazon CloudWatch alarm is configured to invoke my Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling policy, but it's not working.

Short description

When a CloudWatch alarm transitions to a new alarm state, the alarm invokes all configured actions for that alarm state. Amazon EC2 Auto Scaling uses the time period configured on the alarm to determine whether the alarm state should change. However, an alarm invokes an Auto Scaling action once each minute when the alarm remains in a new state, regardless of the configured time period.

The common reasons why a CloudWatch alarm state change doesn't invoke an Amazon EC2 Auto Scaling policy are as follows:

  • The Auto Scaling action isn't set up for the CloudWatch alarm.
  • The scaling policy in the Amazon EC2 Auto Scaling group isn't set up. This prevents the evaluation of the Auto Scaling group.
  • The Auto Scaling group has simple scaling policies or step scaling policies that conflict. This prevents some policies from starting.
  • The Auto Scaling group has an incomplete lifecycle hook. This prevents all simple scaling policies from being applied. A pending instance also causes delays in step and target tracking scaling policies. This is because Auto Scaling doesn't count the instance towards the group's capacity until the lifecycle hook and the warmup time finishes (for scale out). The instance counts towards the group's capacity (for scale in) to prevent overly scaling.
  • The lifecycle hook finishes when it times out or when you call the CompleteLifecycleAction operation or AWS Command Line Interface (AWS CLI).

Resolution

Before you start, verify that your CloudWatch alarm is in the ALARM state. If an alarm's configuration doesn't match the metric threshold that it's monitoring, then the alarm might not transition to the ALARM state. If an alarm doesn't change states, then it doesn't invoke Auto Scaling policies. For more information on how CloudWatch alarms are evaluated, see Evaluating an alarm.

Verify that your CloudWatch alarm enters the ALARM state when expected by checking the alarm's threshold value. Increase or decrease the threshold to match your expected value. Also, review the alarm's period and evaluation period. You might need to edit your alarm's period and evaluation period to invoke your Auto Scaling policy as expected. For more information on verifying alarm actions, see How can I be sure that CloudWatch alarms activate actions?

Important: When creating or editing alarms, keep the following points in mind:

  • Verify that you haven't suspended scaling processes for your Auto Scaling group. If the scaling processes are suspended, then resume the processes.
  • Never create, edit, or delete the alarms associated with target tracking scaling policies. Changing these automatically created alarms might cause unintended effects. The threshold of these alarms is automatically determined based on the target value set in the scaling policy.

Check if Auto Scaling actions are allowed for the CloudWatch alarm

For a CloudWatch alarm to invoke an Auto Scaling policy, turn on the ActionsEnabled parameter in the alarm's configuration. Verify that the ActionsEnabled parameter is set to true in your alarm's configuration.

Note: If you create or update your alarm in the CloudWatch console, then the ActionsEnabled parameter is set to true by default.

To check and allow alarm actions using the AWS CLI, complete the following steps:

  1. Check your configuration with the describe-alarms command.

    aws cloudwatch describe-alarms --alarm-names "myalarm" --query 'MetricAlarms[].ActionsEnabled'

    Note: Replace myalarm with your alarm's ID.

  2. Review the output. If the ActionsEnabled parameter isn't set to true, then turn on alarm actions with the enable-alarm-actions command.

    aws cloudwatch enable-alarm-actions --alarm-names myalarm

Note: If you receive errors when you run AWS CLI commands, then see Troubleshoot AWS CLI errors. Also, verify that you're using the most recent AWS CLI version.

To check and turn on alarm actions with the CloudWatch API, complete the following steps:

  1. Call the DescribeAlarms operation to check your current configuration.
  2. If actions aren't allowed for your alarm, then call the EnableAlarmActions operation to turn on actions.

Check the simple scaling and step scaling policies for your Auto Scaling group

To check your group's scaling policies in the Amazon EC2 console, complete the following steps:

  1. Open the Amazon EC2 console.
  2. In the navigation pane, under Auto Scaling, choose Auto Scaling Groups.
  3. In the content pane, choose your Auto Scaling group.
  4. Choose the Automatic Scaling tab and then take note of the Policy type.

To check scaling policies with the AWS CLI, run the describe-policies command with the --policy-types parameter. To check scaling policies with an AWS API, call the DescribePolicies operation with the PolicyTypes parameter. The output lists each policy type.

If you have one simple scaling policy in effect, then any other simple scaling policies aren't invoked until the following conditions are met:

  • The simple scaling policy in effect is complete.
  • The cooldown period for the Auto Scaling policy elapsed. A simple scaling policy honors the Auto Scaling policy's default or specified cooldown period.

Note: Running a simple scaling policy doesn't completely block the step scaling or target tracking scaling policies. Be sure that contradictory policies aren't applied at the same time.

Check for Auto Scaling lifecycle hooks in your Auto Scaling policy

A simple scaling policy's scaling activities pause when an Auto Scaling lifecycle hook is running. However, a step scaling policy still runs if a lifecycle hook is running.

Check that all lifecycle hooks finish with a CONTINUE or ABANDON result after the global timeout period or heartbeat timeout period ends.

To check for lifecycle hook actions in the Amazon EC2 console, complete the following steps:

  1. Open the Amazon EC2 console.
  2. In the navigation pane, under Auto Scaling, choose Auto Scaling Groups.
  3. In the content pane, choose your Auto Scaling group.
  4. Choose the Activity tab and then scroll to the Activity history section.
  5. Review the activity for any ongoing lifecycle hook actions.

For steps to complete a lifecycle hook, see Complete a lifecycle action.

Related information

How do I troubleshoot scaling issues with my Amazon EC2 Auto Scaling group?

Troubleshoot Amazon EC2 Auto Scaling

AWS OFFICIAL
AWS OFFICIALUpdated 16 days ago