What's the max rate limit of s3 bucket access?


5500 requests per sec seems not working any more. I'm writing a script for finding misconfigured s3 instance. Like this: r = requests.get(f"http://{bucketname}.s3.amazonaws.com") When my script reached 100k requests in 1 hour, all response code received turned into 404. What should I do with my code?

asked 6 months ago4169 views
2 Answers

I'd suggest you to go through this re:Post Knowledge Center Article

Same topic is discussed in this re:Post answer, where it's discussed that If there is a fast spike in the request rate for objects in a prefix, Amazon S3 might return 503 Slow Down errors while it scales in the background to handle the increased request rate. To avoid these errors, you can configure your application to gradually increase the request rate and retry failed requests using an exponential backoff algorithm.

Additionally take a look at Optimizing S3 Performance

Hope this helps.


profile pictureAWS
answered 6 months ago
  • I didn't encounter 503, it's just 404 instead of 403 (if bucket exists)


Hello, Amazon S3 has a limit of 5500 requests per second per prefix. Besides, there is a service quota limit per account. When you exceed the service quota, S3 got throttled. You can request a service quota increase. For further information you may visit the AWS documentation [https://docs.aws.amazon.com/general/latest/gr/s3.html#limits_s3].

answered 6 months ago
  • I scan s3 as anonymous user, so there is no quota increase for me. And 5500 is not in my case.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions