An MDP-based Recommender System

Abstract

Typical Recommender systems adopt a static view of the recommendation process and treat it as a prediction problem. We argue that it is more appropriate to view the problem of generating recommendations as a sequential decision problem and, consequently, that Markov decision processes (MDP) provide a more appropriate model for Recommender systems. MDPs introduce two benefits: they take into account the long-term effects of each recommendation, and they take into account the expected value of each recommendation. To succeed in practice, an MDP-based Recommender system must employ a strong initial model; and the bulk of this paper is concerned with the generation of such a model. In particular, we suggest the use of an n-gram predictive model for generating the initial MDP. Our n-gram model induces a Markov-chain model of user behavior whose predictive accuracy is greater than that of existing predictive models. We describe our predictive model in detail and evaluate its performance on real data. In addition, we show how the model can be used in an MDP-based Recommender system.

Appears in Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI2002)

References in corpus (3)

• Empirical Analysis of Predictive Algorithms for Collaborative Filtering
• Using Temporal Data for Making Recommendations
• Anytime State-Based Solution Methods for Decision Processes with non-Markovian Rewards

Cited by in corpus (10)

• Personalized Course Sequence Recommendations
• GAG: Global Attributed Graph Neural Network for Streaming Session-based Recommendation
• Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation
• A probabilistic model to resolve diversity-accuracy challenge of recommendation systems
• Collaborative Filtering with Recurrent Neural Networks
• Distributed Online Learning in Social Recommender Systems
• Maximum Entropy for Collaborative Filtering
• Breaking the Softmax Bottleneck for Sequential Recommender Systems with Dropout and Decoupling
• Context-aware short-term interest first model for session-based recommendation
• Sequential Relevance Maximization with Binary Feedback