API Gateway Throttling Behavior Not Matching Configured QPS

API Gateway's throttling behavior can sometimes appear inconsistent with configured QPS limits due to several factors. While API Gateway generally enforces throttling limits, the observed behavior you're experiencing is not uncommon and can be attributed to a few key aspects of how API Gateway operates:

1. API Gateway does not typically fail in enforcing QPS limits, but the implementation details can lead to apparent discrepancies.

2. Several factors contribute to the observed throttling behavior:

   a) Burst capacity: API Gateway uses a token bucket algorithm for throttling. This allows for short bursts of traffic above the steady-state rate, which could explain why you're seeing more requests than expected passing through initially.

   b) Account-level limits: There are default account-level throttling limits that apply across all APIs in a region. These might be higher than your specific API's limits, allowing more requests through before throttling occurs.

   c) Throttling order: API Gateway applies throttling settings in a specific order: per-client or per-method limits in a usage plan, per-method limits for an API stage, account-level throttling, and finally AWS regional throttling. This layered approach can sometimes result in more requests being allowed than expected at the individual API level.

   d) Regional variations: Some AWS regions have different default throttle and burst quotas, which could affect the observed behavior.

3. To ensure stricter enforcement of QPS limits:

   a) Review and adjust your usage plan settings, ensuring they align with your desired QPS.

   b) Consider implementing client-side throttling to complement server-side controls.

   c) Monitor your API closely using CloudWatch metrics to understand actual request patterns and adjust accordingly.

   d) If necessary, contact AWS support to discuss your specific use case and potentially adjust account-level quotas.

It's important to note that while API Gateway provides powerful throttling capabilities, achieving exact QPS limits in real-time can be challenging due to the distributed nature of the service and the various levels at which throttling is applied. Regular monitoring and fine-tuning of your throttling settings may be necessary to achieve the desired behavior for your specific use case.
Sources
Throttle requests to your REST APIs for better throughput in API Gateway - Amazon API Gateway
Throttle requests to your HTTP APIs for better throughput in API Gateway - Amazon API Gateway
API Gateway throttling discrepancy | AWS re:Post

Hi,

Are you sure that you did set the QPS at the right level? API Gateway allows for throttling at different level: see https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html#apigateway-how-throttling-limits-are-applied

API Gateway applies your throttling-related settings in the following order:
Per-client or per-method throttling limits that you set for an API stage in a usage plan
Per-method throttling limits that you set for an API stage
Account-level throttling per Region
AWS Regional throttling

If you apply you QPS limit to the wrong level, the total of queries reaching the gateway may be superior to what you want,

Best,

Didier