How do I resolve Auto Scaling issues in Amazon Keyspaces?

Short description

Amazon Keyspaces uses the target tracking AWS Application Auto Scaling. The target utilization percentage value is the ratio of consumed capacity to provisioned capacity.

Application Auto Scaling creates Amazon CloudWatch alarms on your behalf. This alarm invokes AWS Application Auto Scaling that notifies Amazon Keyspaces to adjust the table's provisioned throughput capacity.

Application Auto Scaling for Amazon Keyspaces modifies provisioned throughput only when the workload stays reduced or elevated for several minutes. For example, you set the minimum read capacity units (RCUs) to 100 and target utilization to 70%:

• Application Auto Scaling increases the provisioned capacity when utilization exceeds 70 RCUs for at least three consecutive minutes.
• Application Auto Scaling decreases provisioned capacity when utilization is 20% or more below the target for 15 consecutive minutes (50 RCUs).

Resolution

Application Auto Scaling might not be suitable for running spiky traffic loads on a table. This is because there might not be enough consecutive data points for the consumed capacity to match your capacity needs. In such cases, you might experience throttling on your table because consumed capacity exceeds provisioned capacity. This happens because enough data points didn't breach the utilization threshold for Application Auto Scaling to scale your capacity.

To determine whether consumed capacity is higher than provisioned capacity, use Amazon CloudWatch metrics. Use the Sum statistic to calculate the consumed throughput. For example, get the Sum value over a span of one minute. Then, divide it by 60 to calculate the average ConsumedReadCapacityUnits per second. Note that this average doesn't highlight any large but brief spikes in read activity that occurred during that minute. You can compare the calculated value to the provisioned throughput value that you provide for Amazon Keyspaces.

The following are some best practices to troubleshoot Application Auto Scaling issues on your Amazon Keyspaces table:

• If the table's traffic is frequently unpredictable, consider updating the table to On-demand mode. The on-demand mode is suitable for running spiky workloads because it accommodates to your capacity needs instantly. As and when your traffic becomes consistent or gradually increases, start using Application Auto Scaling again.
• Configure a retry policy in your application. Use exponential backoff, and then retry. Or, add jitter. You can use these approaches to introduce some delay between subsequent retries. With these approaches, you can make sure that the retries aren't exhausted quickly. For more information, see Exponential backoff and jitter.
• Don't delete the CloudWatch alarms that Application Auto Scaling for Amazon Keyspaces creates. If you delete these alarms, Application Auto Scaling might not work as expected. If you accidentally delete these alarms, then turn off Application Auto Scaling, and then turn it on again. CloudWatch automatically recreates the alarms.
• Don't rely on Application Auto Scaling to handle occasional short-duration activity spikes. Application Auto Scaling works best when there are gradual increases or decreases in traffic. The table's built-in burst capacity handles occasional activity spikes. Note that Amazon Keyspaces can also consume burst capacity for background maintenance and other tasks without prior notice.