How to set rate limit and burst in a usage plan in api gateway?

Question

based on the docs, enabling throttling allows to set rate and burst in a usage plan. if burst is the number of request api will handle concurrently, so should the rate be always equal or less than the burst limit? why would we want a higher rate than we can can handle?

Answer

This is related to the token bucket algorithm.

Firstly from the definition [1]: A burst limit represents the maximum number of concurrent requests at any given time, while a rate limit defines the number of requests allowed per second.

Allow me to refer to the below image from [2], and assume x-axis represents 100 ms while y-axis represents the number of requests.

![Enter image description here](/media/postImages/original/IMjVtsAAKYSLmMT4bru7nItw)

In this case, our back-end is designed to handle 30 RPS workload, with the capability to process at most 12 (spiky) requests. Therefore, I would set the rate limit to 30 and the burst limit to 12 accordingly.

Both limits protect my backend from being overloaded.

---

[1] https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html

[2] https://www.geeksforgeeks.org/leaky-bucket-algorithm/

How to set rate limit and burst in a usage plan in api gateway?

Relevant content