What's the max rate limit of s3 bucket access?

Question

5500 requests per sec seems not working any more.
I'm writing a script for finding misconfigured s3 instance.
Like this:
    r = requests.get(f"http://{bucketname}.s3.amazonaws.com")
When my script reached 100k requests in 1 hour, all response code received turned into 404.
What should I do with my code?

Answer

I'd suggest you to go through this [re:Post Knowledge Center Article](https://repost.aws/knowledge-center/http-5xx-errors-s3)

Same topic is discussed in this [re:Post answer](https://repost.aws/questions/QUfZ2g8JGUQbKi92pizkeqtw/s3-putobject-rate-limit-reached), where it's discussed that *If there is a fast spike in the request rate for objects in a prefix, Amazon S3 might return 503 Slow Down errors while it scales in the background to handle the increased request rate. To avoid these errors, you can configure your application to gradually increase the request rate and retry failed requests using an exponential backoff algorithm.*

Additionally take a look at [Optimizing S3 Performance](https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html)

Hope this helps.

Abhishek

Answer

Hello, 
Amazon S3 has a limit of 5500 requests per second per prefix. Besides, there is a service quota limit per account. When you exceed the service quota, S3 got throttled. You can request a service quota increase. For further information you may visit the AWS documentation [https://docs.aws.amazon.com/general/latest/gr/s3.html#limits_s3].

What's the max rate limit of s3 bucket access?

相关内容