Deep reinforcement learning for search, recommendation, and online advertising: a survey

Abstract

Search, recommendation, and online advertising are the three most important information-providing mechanisms on the web. These information seeking techniques, satisfying users' information needs by suggesting users personalized objects (information or services) at the appropriate time and place, play a crucial role in mitigating the information overload problem. With recent great advances in deep reinforcement learning (DRL), there have been increasing interests in developing DRL based information seeking techniques. These DRL based techniques have two key advantages -- (1) they are able to continuously update information seeking strategies according to users' real-time feedback, and (2) they can maximize the expected cumulative long-term reward from users where reward has different definitions according to information seeking applications such as click-through rate, revenue, user satisfaction and engagement. In this paper, we give an overview of deep reinforcement learning for search, recommendation, and online advertising from methodologies to applications, review representative algorithms, and discuss some appealing research directions.

References in corpus (24)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Deep Reinforcement Learning for Page-wise Recommendations
• Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
• Deep Reinforcement Learning in Large Discrete Action Spaces
• Real-Time Bidding by Reinforcement Learning in Display Advertising
• Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
• An Efficient Bandit Algorithm for Realtime Multivariate Optimization
• Cascading Bandits: Learning to Rank in the Cascade Model
• Deep Reinforcement Learning for List-wise Recommendations
• Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising
• Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling
• Learning Contextual Bandits in a Non-stationary Environment
• Reinforcement Learning based Recommender System using Biclustering Technique
• RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising
• DCM Bandits: Learning to Rank with Multiple Clicks
• Whole-Chain Recommendations
• Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application
• Task-Oriented Query Reformulation with Reinforcement Learning
• Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learning
• Large-scale Interactive Recommendation with Tree-structured Policy Gradient
• A Multi-Agent Reinforcement Learning Method for Impression Allocation in Online Display Advertising
• Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
• Bernoulli Rank-$1$ Bandits for Click Feedback
• Adaptive Keywords Extraction with Contextual Bandits for Advertising on Parked Domains

Cited by in corpus (8)

• Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems
• Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities
• Toward Simulating Environments in Reinforcement Learning Based Recommendations
• Learning to Collaborate in Multi-Module Recommendation via Multi-Agent Reinforcement Learning without Communication
• AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation
• The Big Three: A Methodology to Increase Data Science ROI by Answering the Questions Companies Care About
• Sequential Search with Off-Policy Reinforcement Learning
• Beyond Our Behavior: The GDPR and Humanistic Personalization