According to the documentation, with exploration weight as '0', there should be no exploration and the items returned should be based on relevance. With our testing we don't have the items returned completely based on relevance, it still returns popular items which the user has not interacted with at all. Our dataset has 50 users (with 100 interactions each) and also some 17 anonymous interactions (with no user_id and only session_id).
Is this an issue with the dataset somehow?
This is our user-item interaction schema example:
"type": "record",
"name": "Interactions",
"namespace": "com.amazonaws.personalize.schema",
"fields": [
{
"name": "USER_ID",
"type": "string"
},
{
"name": "ITEM_ID",
"type": "string"
},
{
"name": "EVENT_TYPE",
"type": "string"
},
{
"name": "DEVICE_TYPE",
"type": "string"
},
{
"name": "LOCATION",
"type": "string"
},
{
"name": "SESSION_ID",
"type": [
"null",
"string"
]
},
{
"name": "TIMESTAMP",
"type": "long"
}
]
}
5K interactions for 50 users is an extremely small dataset. How many distinct items are in the dataset? What is the distribution of interaction counts across those items? What are the offline metrics from the model?