Neural Interactive Collaborative Filtering

Abstract

In this paper, we study collaborative filtering in an interactive setting, in which the recommender agents iterate between making recommendations and updating the user profile based on the interactive feedback. The most challenging problem in this scenario is how to suggest items when the user profile has not been well established, i.e., recommend for cold-start users or warm-start users with taste drifting. Existing approaches either rely on overly pessimistic linear exploration strategy or adopt meta-learning based algorithms in a full exploitation way. In this work, to quickly catch up with the user's interests, we propose to represent the exploration policy with a neural network and directly learn it from the feedback data. Specifically, the exploration policy is encoded in the weights of multi-channel stacked self-attention neural networks and trained with efficient Q-learning by maximizing users' overall satisfaction in the recommender systems. The key insight is that the satisfied recommendations triggered by the exploration recommendation can be viewed as the exploration bonus (delayed reward) for its contribution on improving the quality of the user profile. Therefore, the proposed exploration policy, to balance between learning the user profile and making accurate recommendations, can be directly optimized by maximizing users' long-term satisfaction with reinforcement learning. Extensive experiments and analysis conducted on three benchmark collaborative filtering datasets have demonstrated the advantage of our method over state-of-the-art methods.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Playing Atari with Deep Reinforcement Learning
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• A Contextual-Bandit Approach to Personalized News Article Recommendation
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Deep Reinforcement Learning for Page-wise Recommendations
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
• Offline A/B testing for Recommender Systems

Cited by in corpus (19)

• Advances and Challenges in Conversational Recommender Systems: A Survey
• LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems
• A Survey on Reinforcement Learning for Recommender Systems
• Jointly Learning to Recommend and Advertise
• Whole-Chain Recommendations
• Towards Explainable Conversational Recommender Systems
• Exploring ChatGPT for Next-generation Information Retrieval: Opportunities and Challenges
• User Retention-oriented Recommendation with Decision Transformer
• DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation
• Dual Policy Learning for Aggregation Optimization in Graph Neural Network-based Recommender Systems
• Beyond Single Items: Exploring User Preferences in Item Sets with the Conversational Playlist Curation Dataset
• Towards Validating Long-Term User Feedbacks in Interactive Recommendation Systems
• AutoLoss: Automated Loss Function Search in Recommendations
• Approximated Doubly Robust Search Relevance Estimation
• Pre-trained Language Model for Web-scale Retrieval in Baidu Search
• Towards Two-Stage Counterfactual Learning to Rank
• Pre-trained Language Model based Ranking in Baidu Search
• Inferring Substitutable and Complementary Products with Knowledge-Aware Path Reasoning based on Dynamic Policy Network
• Data-Efficient Reinforcement Learning for Malaria Control