High Quality Related Search Query Suggestions using Deep Reinforcement Learning

Abstract

"High Quality Related Search Query Suggestions" task aims at recommending search queries which are real, accurate, diverse, relevant and engaging. Obtaining large amounts of query-quality human annotations is expensive. Prior work on supervised query suggestion models suffered from selection and exposure bias, and relied on sparse and noisy immediate user-feedback (e.g., clicks), leading to low quality suggestions. Reinforcement Learning techniques employed to reformulate a query using terms from search results, have limited scalability to large-scale industry applications. To recommend high quality related search queries, we train a Deep Reinforcement Learning model to predict the query a user would enter next. The reward signal is composed of long-term session-based user feedback, syntactic relatedness and estimated naturalness of generated query. Over the baseline supervised model, our proposed approach achieves a significant relative improvement in terms of recommendation diversity (3%), down-stream user-engagement (4.2%) and per-sentence word repetitions (82%).

arXiv commentMulti-Armed Bandits and Reinforcement Learning: Advancing Decision Making in E-Commerce and Beyond at KDD 2021

References in corpus (7)

• Language Models are Few-Shot Learners
• Tune: A Research Platform for Distributed Model Selection and Training
• Deep Reinforcement Learning for Dialogue Generation
• Learning to summarize from human feedback
• Deep Reinforced Query Reformulation for Information Retrieval
• Domain-Independent turn-level Dialogue Quality Evaluation via User Satisfaction Estimation
• Query Understanding for Natural Language Enterprise Search