Generative Adversarial User Model for Reinforcement Learning Based Recommendation System

Abstract

There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a novel model-based reinforcement learning framework for recommendation systems, where we develop a generative adversarial network to imitate user behavior dynamics and learn her reward function. Using this user model as the simulation environment, we develop a novel Cascading DQN algorithm to obtain a combinatorial recommendation policy which can handle a large number of candidate items efficiently. In our experiments with real data, we show this generative adversarial user model can better explain user behavior than alternatives, and the RL policy based on this model can lead to a better long-term reward for the user and higher click rate for the system.

Cited by in corpus (19)

• Advances and Challenges in Conversational Recommender Systems: A Survey
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Jointly Learning to Recommend and Advertise
• Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble
• Reinforcement Learning Applications
• Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
• A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Self-Supervised Reinforcement Learning for Recommender Systems
• Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
• Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation
• Learning to Recommend via Meta Parameter Partition
• Deep Reinforcement Learning-Based Product Recommender for Online Advertising
• Generative Inverse Deep Reinforcement Learning for Online Recommendation
• SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing
• Inferring Substitutable and Complementary Products with Knowledge-Aware Path Reasoning based on Dynamic Policy Network
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning
• MBCAL: Sample Efficient and Variance Reduced Reinforcement Learning for Recommender Systems
• Generation of Traffic Flows in Multi-Agent Traffic Simulation with Agent Behavior Model based on Deep Reinforcement Learning