Has anybody noticed Challenge in WAF has become less effective?

Based on the information available, there's no documented change in the effectiveness of AWS WAF Challenge actions in recent days.

The Challenge action in AWS WAF is designed to run silent browser interrogations in the background to verify that requests are coming from legitimate browser sessions rather than automated tools. While it's effective against many bots because they typically can't process the challenge as a legitimate web browser would, it's not as robust as CAPTCHA.

CAPTCHA provides a stronger protection layer by requiring human interaction to solve puzzles, making it more effective against sophisticated bots. However, this comes with tradeoffs - CAPTCHA affects user experience more directly and typically involves higher costs than the Challenge option.

It's worth noting that bot developers continuously work to circumvent protection mechanisms. Even CAPTCHA puzzles have limitations - many have been solved using machine learning and artificial intelligence, and some organizations supplement automated techniques with human intervention to bypass them.

If you've observed decreased effectiveness with Challenge actions, switching to CAPTCHA is a reasonable step for stronger protection, though it will impact user experience more significantly. You might also want to consider using AWS WAF's Bot Control managed rules group for more robust and flexible bot mitigation if the issue persists.
Sources
How to mitigate bot traffic by implementing Challenge actions in your AWS WAF custom rules | AWS Security Blog
CAPTCHA and Challenge in AWS WAF - AWS WAF, AWS Firewall Manager, AWS Shield Advanced, and AWS Shield network security director
AWS WAF CAPTCHA puzzles - AWS WAF, AWS Firewall Manager, AWS Shield Advanced, and AWS Shield network security director
ChallengeAction - AWS WAFV2