Locality-Sensitive Experience Replay for Online Recommendation

Abstract

Online recommendation requires handling rapidly changing user preferences. Deep reinforcement learning (DRL) is gaining interest as an effective means of capturing users' dynamic interest during interactions with recommender systems. However, it is challenging to train a DRL agent, due to large state space (e.g., user-item rating matrix and user profiles), action space (e.g., candidate items), and sparse rewards. Existing studies encourage the agent to learn from past experience via experience replay (ER). They adapt poorly to the complex environment of online recommender systems and are inefficient in determining an optimal strategy from past experience. To address these issues, we design a novel state-aware experience replay model, which uses locality-sensitive hashing to map high dimensional data into low-dimensional representations and a prioritized reward-driven strategy to replay more valuable experience at a higher chance. Our model can selectively pick the most relevant and salient experiences and recommend the agent with the optimal policy. Experiments on three online simulation platforms demonstrate our model' feasibility and superiority toseveral existing experience replay methods.

References in corpus (15)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Addressing Function Approximation Error in Actor-Critic Methods
• Distributed Prioritized Experience Replay
• Deep Reinforcement Learning for Page-wise Recommendations
• Hindsight Experience Replay
• Jointly Learning to Recommend and Advertise
• RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising
• RecSim: A Configurable Simulation Platform for Recommender Systems
• Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Experience Replay Optimization
• Remember and Forget for Experience Replay
• Dynamic Experience Replay