Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning

Abstract

Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedback. Users' feedback can be positive and negative and both types of feedback have great potentials to boost recommendations. However, the number of negative feedback is much larger than that of positive one; thus incorporating them simultaneously is challenging since positive feedback could be buried by negative one. In this paper, we develop a novel approach to incorporate them into the proposed deep recommender system (DEERS) framework. The experimental results based on real-world e-commerce data demonstrate the effectiveness of the proposed framework. Further experiments have been conducted to understand the importance of both positive and negative feedback in recommendations.

arXiv commentarXiv admin note: substantial text overlap with arXiv:1801.00209

References in corpus (7)

• Playing Atari with Deep Reinforcement Learning
• Empirical Analysis of Predictive Algorithms for Collaborative Filtering
• Session-based Recommendations with Recurrent Neural Networks
• Deep Reinforcement Learning for Page-wise Recommendations
• PEGASUS: A Policy Search Method for Large MDPs and POMDPs
• Deep Reinforcement Learning for List-wise Recommendations
• Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions

Cited by in corpus (75)

• Deep Learning based Recommender System: A Survey and New Perspectives
• Deep Reinforcement Learning for Page-wise Recommendations
• Advances and Challenges in Conversational Recommender Systems: A Survey
• Towards Long-term Fairness in Recommendation
• KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems
• Neural Interactive Collaborative Filtering
• Deep Reinforcement Learning for List-wise Recommendations
• Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data
• LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems
• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
• A Survey on Reinforcement Learning for Recommender Systems
• Deep reinforcement learning for search, recommendation, and online advertising: a survey
• Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling
• Jointly Learning to Recommend and Advertise
• Toward Pareto Efficient Fairness-Utility Trade-off inRecommendation through Reinforcement Learning
• Multi-Behavior Graph Neural Networks for Recommender System
• Graph Neural Networks for Social Recommendation
• HAMUR: Hyper Adapter for Multi-Domain Recommendation
• AutoDenoise: Automatic Data Instance Denoising for Recommendations
• Whole-Chain Recommendations
• Reinforcement Learning Applications
• Multi-Task Recommendations with Reinforcement Learning
• Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation
• Exploration and Regularization of the Latent Action Space in Recommendation
• SIGformer: Sign-aware Graph Transformer for Recommendation
• User Retention-oriented Recommendation with Decision Transformer
• Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term Retention
• Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems
• Value Penalized Q-Learning for Recommender Systems
• User Tampering in Reinforcement Learning Recommender Systems
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Toward Simulating Environments in Reinforcement Learning Based Recommendations
• Self-Supervised Reinforcement Learning for Recommender Systems
• Adapting Job Recommendations to User Preference Drift with Behavioral-Semantic Fusion Learning
• Asking Clarifying Questions Based on Negative Feedback in Conversational Search
• Value-aware Recommendation based on Reinforced Profit Maximization in E-commerce Systems
• State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study
• Learning from Negative User Feedback and Measuring Responsiveness for Sequential Recommenders
• Automated Design and Optimization of Distributed Filtering Circuits via Reinforcement Learning
• Future Impact Decomposition in Request-level Recommendations
• Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
• AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation
• Modeling User Retention through Generative Flow Networks
• Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
• Attacking Black-box Recommendations via Copying Cross-domain User Profiles
• Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning
• Deep Reinforcement Learning based Group Recommender System
• AutoLoss: Automated Loss Function Search in Recommendations
• Towards Validating Long-Term User Feedbacks in Interactive Recommendation Systems
• Recommendation Fairness: From Static to Dynamic
• Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems
• Denoising User-aware Memory Network for Recommendation
• An End-to-End Deep RL Framework for Task Arrangement in Crowdsourcing Platforms
• Interactive Recommender System via Knowledge Graph-enhanced Reinforcement Learning
• When Collaborative Filtering Meets Reinforcement Learning
• Rethinking Conversational Recommendations: Is Decision Tree All You Need?
• The Disruptions of 5G on Data-driven Technologies and Applications
• Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation
• Developing Multi-Task Recommendations with Long-Term Rewards via Policy Distilled Reinforcement Learning
• Deep Hierarchical Reinforcement Learning Based Recommendations via Multi-goals Abstraction
• Empowering Denoising Sequential Recommendation with Large Language Model Embeddings
• SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation
• Prompt Tuning as User Inherent Profile Inference Machine
• Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning
• A Load Balanced Recommendation Approach
• Optimized Recommender Systems with Deep Reinforcement Learning
• Off-policy Learning for Multiple Loggers
• CARL: Aggregated Search with Context-Aware Module Embedding Learning
• Surveys without Questions: A Reinforcement Learning Approach
• Value Function Decomposition in Markov Recommendation Process
• D2RLIR : an improved and diversified ranking function in interactive recommendation systems based on deep reinforcement learning
• Simultaneous Relevance and Diversity: A New Recommendation Inference Approach
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning
• Adaptive Neural Architectures for Recommender Systems
• MBCAL: Sample Efficient and Variance Reduced Reinforcement Learning for Recommender Systems