AWS WAF to block automated bot from stealing content.

Question

Ciao

I am trying to use bot control to block bot and content scrapper from stealing content on client blog. After setting up WAF i realized some bot still are still stealing content and google news not working.  I am using an ALB.
If i turn on Anonymous IP List, it blocks the bot but also block VPN and Proxies.

Please how can i go about it, so it doesn't block service i need and use, but block automated bot from stealing content.

Best Regards

Answer

Hello,

You may want to check out some of our WAF Bot Control example rules in the documentation.  One specific rule I think you are looking for is `Explicitly Allow Verified Bots`.  Here is the documentation to set up this rule: https://docs.aws.amazon.com/waf/latest/developerguide/waf-bot-control-example-allow-verified-bots.html

Here is a list of example Bot Control Rules you can implement: https://docs.aws.amazon.com/waf/latest/developerguide/waf-bot-control-examples.html

AWS WAF to block automated bot from stealing content.

Relevant questions