Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation

Abstract

We propose a method to efficiently learn diverse strategies in reinforcement learning for query reformulation in the tasks of document retrieval and question answering. In the proposed framework an agent consists of multiple specialized sub-agents and a meta-agent that learns to aggregate the answers from sub-agents to produce a final answer. Sub-agents are trained on disjoint partitions of the training data, while the meta-agent is trained on the full training set. Our method makes learning faster, because it is highly parallelizable, and has better generalization performance than strong baselines, such as an ensemble of agents trained on the full data. We show that the improved performance is due to the increased diversity of reformulation strategies.

References in corpus (14)

• Efficient Estimation of Word Representations in Vector Space
• Distilling the Knowledge in a Neural Network
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Bidirectional Attention Flow for Machine Comprehension
• A Deep Reinforced Model for Abstractive Summarization
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• A Simple, Fast Diverse Decoding Algorithm for Neural Generation
• An Actor-Critic Algorithm for Sequence Prediction
• A Deep Reinforcement Learning Chatbot
• Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering
• Large scale distributed neural network training through online distillation
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering
• Task-Oriented Query Reformulation with Reinforcement Learning
• Analyzing Language Learned by an Active Question Answering Agent

Cited by in corpus (4)

• Deep reinforcement learning for search, recommendation, and online advertising: a survey
• Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
• Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System
• Reinforcement Learning-driven Information Seeking: A Quantum Probabilistic Approach