My Auto Scaling API calls are getting throttled. What can I do to avoid this?

3 minute read
0

My application receives "Rate Exceeded" errors when calling to Amazon EC2 Auto Scaling, AWS Auto Scaling, or AWS Application Auto Scaling. What can I do to avoid this error?

Short description

All API calls can't exceed the maximum allowed API request rate per account and per Region. This includes API calls from the AWS Command Line Interface (AWS CLI) and the AWS Management Console. If API requests exceed the maximum rate, then you receive a "Rate Exceeded" error, and further API calls are throttled.

Amazon EC2 Auto Scaling, AWS Auto Scaling, and AWS Application Auto Scaling each have their own API throttle buckets. This means that all Amazon EC2 Auto Scaling API calls have a single, shared API limit. Amazon EC2 Auto Scaling API calls don't affect the limit for AWS Application Auto Scaling APIs.

To avoid the "Rate Exceeded" error and throttling, verify that your application is making only necessary calls.

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

Resolution

To prevent or mitigate "Rate Exceeded" errors and throttling, try these solutions:

Validate "describe" calls

Excessive "describe" calls contribute to the total API requests measured against the allowed request rate. Verify your application to be sure that all "describe" calls are necessary, and consider solutions other than "describe" calls where possible. Consider using push notifications from Amazon EventBridge that are sent when instances change state, such as when they start a lifecycle hook.

Check calls from third-party applications

Third-party applications might make continuous calls to Auto Scaling in AWS. Verify your third-party applications to be sure that they're not making unnecessary calls.

Implement error retries and exponential backoffs

Error retries and exponential backoffs can help limit the rate of API calls. Each AWS SDK implements automatic retry logic and exponential backoff algorithms. For more information, see Error Retries and exponential backoff in AWS.

Request a service quota increase in the AWS Support Center

To get a service quota increase, you must confirm that you validated your API call rate, as well as implemented error retries or exponential backoff methods. In your request, you must also provide the Region and timeframe related to the throttling issues.

Avoid bursts of activity

Avoid situations that cause bursts of API calls. For example, don't set all instances in an Amazon EC2 Auto Scaling group enable scale in protection when you launch them. Instead, enable this option by default on the group so that all instances have protection enabled by default.


Related information

Exponential backoff and jitter

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago