Reward Constrained Interactive Recommendation with Natural Language Feedback
arXiv:2005.01618
Abstract
Text-based interactive recommendation provides richer user feedback and has demonstrated advantages over traditional interactive recommender systems. However, recommendations can easily violate preferences of users from their past natural-language feedback, since the recommender needs to explore new items for further improvement. To alleviate this issue, we propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference, which is incorporated into the standard RL objective of maximizing expected cumulative future rewards. Our proposed framework is general and is further extended to the task of constrained text generation. Empirical results show that the proposed method yields consistent improvement relative to standard RL methods.
Appeared in NeurIPS 2019; Updated version
References in corpus (14)
- A Contextual-Bandit Approach to Personalized News Article Recommendation
- DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
- Safe Exploration in Continuous Action Spaces
- Reward Constrained Policy Optimization
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
- Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
- Long Text Generation via Adversarial Training with Leaked Information
- Adversarial Ranking for Language Generation
- Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
- Policy Gradients with Variance Related Risk Criteria
- Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
- Topic-Guided Variational Autoencoders for Text Generation
- Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback
- Learning Compressed Sentence Representations for On-Device Text Processing