Open-Retrieval Conversational Question Answering

Abstract

Conversational search is one of the ultimate goals of information retrieval. Recent research approaches conversational search by simplified settings of response ranking and conversational question answering, where an answer is either selected from a given candidate set or extracted from a given passage. These simplifications neglect the fundamental role of retrieval in conversational search. To address this limitation, we introduce an open-retrieval conversational question answering (ORConvQA) setting, where we learn to retrieve evidence from a large collection before extracting answers, as a further step towards building functional conversational search systems. We create a dataset, OR-QuAC, to facilitate research on ORConvQA. We build an end-to-end system for ORConvQA, featuring a retriever, a reranker, and a reader that are all based on Transformers. Our extensive experiments on OR-QuAC demonstrate that a learnable retriever is crucial for ORConvQA. We further show that our system can make a substantial improvement when we enable history modeling in all system components. Moreover, we show that the reranker component contributes to the model performance by providing a regularization effect. Finally, further in-depth analyses are performed to provide new insights into ORConvQA.

Accepted to SIGIR'20

References in corpus (17)

• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)
• Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems
• BERT with History Answer Embedding for Conversational Question Answering
• Quasar: Datasets for Question Answering by Search and Reading
• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
• Analyzing and Characterizing User Intent in Information-seeking Conversations
• FlowQA: Grasping Flow in History for Conversational Machine Comprehension
• User Intent Prediction in Information-seeking Conversations
• Conversational Product Search Based on Negative Feedback
• Attentive History Selection for Conversational Question Answering
• Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
• Neural Matching Models for Question Retrieval and Next Question Prediction in Conversation
• GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
• Answer Interaction in Non-factoid Question Answering Systems
• ReQA: An Evaluation for End-to-End Answer Retrieval Models

Cited by in corpus (8)

• Few-Shot Conversational Dense Retrieval
• Query Performance Prediction: From Ad-hoc to Conversational Search
• Passage Retrieval for Outside-Knowledge Visual Question Answering
• Asking Clarifying Questions Based on Negative Feedback in Conversational Search
• P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuning
• ProCIS: A Benchmark for Proactive Retrieval in Conversations
• Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts
• Improving Conversational Passage Re-ranking with View Ensemble