AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.
Is Amazon Personalize Suitable for Infinite Feed Recommendation in Social Media App?
We are developing a social media app with an exploring feed, comparable to Instagram, TikTok or Reddit. Currently, our post recommendation algorithm is based on SQL functions to recommend posts based on the user's recent weighted interactions (views, likes, comments, etc.) with posts, taking into account the post tags. The post recommendations are paginated and filtered to exclude posts that the user has already seen. Although this solution provides good results, it is not scalable. Therefore, we are considering migrating to a machine learning-based recommendation engine once we have reached a critical mass of items and interaction data.
As a small team with limited ML experience but substantial experience with Amazon services in our application, we are considering using Amazon Personalize as our recommendation engine. However, I have come across discussions in forums and on Stack Overflow regarding the limitations of the service in the context of infinite feeds, particularly in terms of pagination and filtering of seen posts.
Therefore, I would like to pose the question to the community for discussion: Is Amazon Personalize suitable for such a use case, and what considerations should be taken into account when implementing it?
- Topics
- Machine Learning & AI
- Language
- English
asked 2 years ago586 views
- Newest
- Most votes
- Most comments
I would tentatively say it's not ideal by itself - but if you're approaching such a use-case from scratch, you could use it for a quick start and backfill with other more basic methods (like your current SQL).
One reason (it sounds like you might be aware) is that the GetRecommendations API can return at most 500 results, which may already easily be enough for users in some "infinite" scroll applications, but not for others - depending on typical user dwell time/engagement. You could PutEvents feedback while the user scrolls through the the initial results, and then fetch more recommendations when the initial set is exhausted - but I suspect the likelihood of repeat items would be high unless you logged every view event and applied a filter expression. This would be quite intensive on the model so I could see a risk of running in to quota issues at scale. Alternatively you could just default to some more basic system when the initial Personalize records are exhausted, and at least benefit from ML personalization of your initial feed.
For use-cases like these, the freshness of the feed is typically very important, so you may want to end up filtering out every previously-seen item anyway... But maybe it's different for you.
Another consideration to watch out for is that although Personalize allows you to submit a value for each event (e.g. view duration), this is currently used as a data selection threshold in training rather than a smooth signal: so you'd still need to decide how long a user should dwell on a story/item before marking it as a successful event.
So I would not expect the service (as it is today) to be the most perfect/optimal recommender possible for long-session, infinite feeds... BUT, if the team doesn't already have deep expertise in the range of DL/RL/GNN technologies that might be important to engineer something more specific, and you think your use-case would fit reasonably within the quota limits - then maybe it's worth plugging in for a quick uplift before revisiting the challenge in future?
