How can we discourage Personalize from over-focussing on popular items?

Question

I've seen a situation (in retail) where Amazon Personalize re-ranking and user-personalization solutions seem to be over-indexing on recommending the most popular items in the catalogue, even for users with different tastes and a decent amount of interaction history (results are not exactly the same, but popular items seem very heavily weighted).

These recipes don't seem to have the popularity_discount_factor hyperparameter like SIMS does... So are there any particular suggestions for reducing or avoiding this effect?

These recipes don't seem to have the `popularity_discount_factor` hyperparameter like [SIMS does](https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-sims.html)... So are there any particular suggestions for reducing or avoiding this effect?

Answer

Hi,

Popular items have more interactions, which can mean that the model has more information on them and can lead to them being recommended more often if you have a long tail of items in your catalog that do not have any or have only very few interactions.

Some things you can do:
* Apply a [promotion](https://aws.amazon.com/blogs/machine-learning/customize-your-recommendations-by-promoting-specific-items-using-business-rules-with-amazon-personalize/) to have a certain percentage of your recommendation be of a filtered type, for instance "promoted" items or items that are new to the catalog.
* [Increase the exploration_weight](https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-new-item-USER_PERSONALIZATION.html):  this will determines how frequently recommendations include items with less interactions data or relevance.
* [Reduce the value bptt](https://docs.aws.amazon.com/personalize/latest/dg/native-recipe-new-item-USER_PERSONALIZATION.html), this will increase the effect of newer interactions (or increase it to take into account the longer interactions history).
* [Perform HPO](https://docs.aws.amazon.com/personalize/latest/dg/customizing-solution-config-hpo.html)(if you haven't already)
* Try to use a longer timeframe of historical data - this can mean collecting more data. More data, will lead to a richer model.
* Review your [user](https://docs.aws.amazon.com/personalize/latest/dg/user-dataset-requirements.html) and [item](https://docs.aws.amazon.com/personalize/latest/dg/item-dataset-requirements.html) metadata. You may have do some feature engineering to make sure the metadata you are sending is 1. in the correct format, 2. you have all relevant fields and 3. the fields you have added value to the model. Make sure that data that is "categorical" is marked as "categorical: true".

How can we discourage Personalize from over-focussing on popular items?

Relevant content