Deep Reinforcement Learning for List-wise Recommendations

Abstract

Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedbacks. In particular, we introduce an online user-agent interacting environment simulator, which can pre-train and evaluate model parameters offline before applying the model online. Moreover, we validate the importance of list-wise recommendations during the interactions between users and agent, and develop a novel approach to incorporate them into the proposed framework LIRD for list-wide recommendations. The experimental results based on a real-world e-commerce dataset demonstrate the effectiveness of the proposed framework.

References in corpus (5)

• Continuous control with deep reinforcement learning
• Empirical Analysis of Predictive Algorithms for Collaborative Filtering
• Deep Reinforcement Learning for Page-wise Recommendations
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
• Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions

Cited by in corpus (39)

• Deep Reinforcement Learning for Page-wise Recommendations
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
• Towards Long-term Fairness in Recommendation
• Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling
• Deep reinforcement learning for search, recommendation, and online advertising: a survey
• Jointly Learning to Recommend and Advertise
• Toward Pareto Efficient Fairness-Utility Trade-off inRecommendation through Reinforcement Learning
• A Survey on Session-based Recommender Systems
• Sequential/Session-based Recommendations: Challenges, Approaches, Applications and Opportunities
• AutoDenoise: Automatic Data Instance Denoising for Recommendations
• Multi-Task Recommendations with Reinforcement Learning
• RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm
• User Retention-oriented Recommendation with Decision Transformer
• Measuring Recommender System Effects with Simulated Users
• Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems
• Deep Reinforcement Learning for Imbalanced Classification
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Toward Simulating Environments in Reinforcement Learning Based Recommendations
• Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems
• Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment
• Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
• FINN.no Slates Dataset: A new Sequential Dataset Logging Interactions, allViewed Items and Click Responses/No-Click for Recommender Systems Research
• AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation
• Deep Trustworthy Knowledge Tracing
• Deep Reinforcement Learning based Group Recommender System
• Deep Reinforcement Learning-Based Product Recommender for Online Advertising
• Learning to Recommend via Meta Parameter Partition
• Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction
• Top-N Recommendation with Counterfactual User Preference Simulation
• Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect
• Generative Inverse Deep Reinforcement Learning for Online Recommendation
• Sequential Search with Off-Policy Reinforcement Learning
• Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation
• Combinatorial Keyword Recommendations for Sponsored Search with Deep Reinforcement Learning
• Deep Hierarchical Reinforcement Learning Based Recommendations via Multi-goals Abstraction
• Off-policy Learning for Multiple Loggers
• A Load Balanced Recommendation Approach
• D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning
• Action-conditional Sequence Modeling for Recommendation