By using AWS re:Post, you agree to the Terms of Use
/Amazonbot web crawler DOS incident/

Amazonbot web crawler DOS incident

0

Has anybody had experience with the Amazonbot web crawler? (especially with it looking like a DOS incident)

We have a client where their service (hosted on AWS) kept going down until we identified an unusual amount of traffic from an Amazonbot with the User-Agent of "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)", after blocking it at an Application Load Balancer level via User-Agent Header rules (since modifying the "robots.txt" did not work / wasn't read) the response time of the clients service dropped significantly and has remained up.

If anybody knows what generally causes Amazonbot to crawl sites, or how I can identify the cause for this specific client's site, that would be extremely useful, cheers.

1 Answers
0

AmazonBot is not a service/feature of AWS.

For any questions or concerns regarding AmazonBot, I recommend reaching out to the AmazonBot team via the Contact link at the bottom of this page: https://developer.amazon.com/support/amazonbot

answered 4 months ago
  • Hi Chris, I followed the link from the user agent with no avail currently, I did send them an email yesterday with more details of the situation as it was happening, although they have still not replied. I guess the best thing to do now is to wait for a response and be glad that the main issue has been fixed.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions