Why is my expected instance count different from the actual instance count that Amazon ECS capacity providers manage?

4 minute read
0

My Amazon Elastic Container Service (Amazon ECS) capacity provider has a different desired instance count than what I expect. Or, the desired instance count is different from what the associated Amazon EC2 Auto Scaling group has.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Check whether instances from the EC2 Auto Scaling group failed to join the cluster

Check whether an instance appears in the Auto Scaling group that's associated with your capacity provider but not on the Amazon ECS console. If there are network configuration, AWS Identity and Access Management (IAM), or Amazon ECS container agent issues, then the container instance can't join the cluster.

To resolve these issues, see Why can't my Amazon EC2 instance join the Amazon ECS cluster?

Check the capacity provider's configurations

Use either the Amazon ECS console or the AWS CLI.

Amazon ECS console

Complete the following steps:

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Clusters.
  3. Select your cluster.
  4. Choose the Infrastructure tab.

AWS CLI

Run the following describe-capacity-providers command:

aws ecs describe-capacity-providers --capacity-providers YOUR_CAPACITY_PROVIDER

Note: Replace YOUR_CAPACITY_PROVIDER with your capacity provider name.

In the output, check the TargetCapacity and minimumScalingStepSize values.

To maintain extra capacity for your clusters, set the value of TargetCapacity to less than 100%. The lower the value, the higher your cluster's extra capacity. For more information about target capacity, see Deep dive on Amazon ECS cluster auto scaling. To modify TargetCapacity, use the Amazon ECS console. Or, run the following update-capacity-provider command:

aws ecs update-capacity-provider --name YOUR_CAPACITY_PROVIDER --auto-scaling-group-provider managedScaling={targetCapacity=DESIRED_VALUE} 

Note: Replace YOUR_CAPACITY_PROVIDER with your capacity provider name and DESIRED_VALUE with your TargetCapacity value.

MinimumScalingStepSize defines the minimum number of instances that Amazon ECS can scale out by. Amazon ECS scales based on MinimumScalingStepSize even when the actual demand is less than the value. As a result, Amazon ECS might temporarily associate more container instances with your capacity provider than are required. To update minimumScalingStepSize, run the following update-capacity-provider command:

aws ecs update-capacity-provider --name YOUR_CAPACITY_PROVIDER --auto-scaling-group-provider managedScaling={minimumScalingStepSize=DESIRED_VALUE}

Note: Replace YOUR_CAPACITY_PROVIDER with your capacity provider name and DESIRED_VALUE with your MinimumScalingStepSize value.

For more information, see Automatically manage Amazon ECS capacity with cluster auto scaling.

Check the Auto Scaling group's configurations

To check the auto scaling policies and capacity configurations for your Auto Scaling group, run the following describe-auto-scaling-groups command:

aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names YOUR_AUTO_SCALING_GROUP

Note: Replace YOUR_AUTO_SCALING_GROUP with the name of the Auto Scaling group associated with your capacity provider.

Amazon ECS cluster auto scaling automatically adds scaling policies to your capacity provider's associated Auto Scaling group. Because of this, it's not a best practice to modify or add scaling policies that are associated with the Auto Scaling group.

To scale Amazon ECS based on metrics such as CPU usage, it's a best practice to use AWS Application Auto Scaling. For information about how to use custom metrics, see Autoscaling Amazon ECS services based on custom metrics with Application Auto Scaling.

Cluster auto scaling doesn't modify the MinimumCapacity or MaximumCapacity values for the Auto Scaling group. If MinimumCapacity is higher than 0, then there might be empty instances. If MaximumCapacity is too low, then the capacity provider can't scale to meet the demands of your Amazon ECS services. To modify these values, run the following update-auto-scaling-group command:

aws autoscaling update-auto-scaling-group --auto-scaling-group-name YOUR_AUTO_SCALING_GROUP --min-size DESIRED_VALUE --max-size DESIRED_VALUE

Note: Replace YOUR_AUTO_SCALING_GROUP with the name of the Auto Scaling group that's associated with your capacity provider and DESIRED_VALUE with your capacity values.

AWS OFFICIAL
AWS OFFICIALUpdated a month ago