A/B testing tools for Amazon Personalize

Yes, this blog post describes methods for A/B testing with Amazon Personalize, provides an architecture diagram, and lists tools built for this purpose.

• https://aws.amazon.com/blogs/machine-learning/using-a-b-testing-to-measure-the-efficacy-of-recommendations-generated-by-amazon-personalize/

For a build path, there are workshops in the Retail Demo Store for A/B, interleaving, and multi-armed bandit testing.

• https://github.com/aws-samples/retail-demo-store/blob/master/workshop/3-Experimentation/3.2-AB-Experiment.ipynb
• https://github.com/aws-samples/retail-demo-store/blob/master/workshop/3-Experimentation/3.3-Interleaving-Experiment.ipynb
• https://github.com/aws-samples/retail-demo-store/blob/master/workshop/3-Experimentation/3.4-Multi-Armed-Bandit-Experiment.ipynb

Update 12/4/2021

Amazon CloudWatch Evidently is a new feature of CloudWatch that supports deploying and measuring A/B tests. It can be used with Amazon Personalize to A/B test Personalize against an existing personalization solution and Personalize or two Personalize model variations.