QuAC : Question Answering in Context

Abstract

We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1) a student who poses a sequence of freeform questions to learn as much as possible about a hidden Wikipedia text, and (2) a teacher who answers the questions by providing short excerpts from the text. QuAC introduces challenges not found in existing machine comprehension datasets: its questions are often more open-ended, unanswerable, or only meaningful within the dialog context, as we show in a detailed qualitative evaluation. We also report results for a number of reference models, including a recently state-of-the-art reading comprehension architecture extended to model dialog context. Our best model underperforms humans by 20 F1, suggesting that there is significant room for future work on this data. Dataset, baseline, and leaderboard available at http://quac.ai.

arXiv commentEMNLP Camera Ready

References in corpus (5)

• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Deep Reinforcement Learning for Dialogue Generation
• Know What You Don't Know: Unanswerable Questions for SQuAD
• NewsQA: A Machine Comprehension Dataset
• Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings

Cited by in corpus (13)

• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
• Query Resolution for Conversational Search with Limited Supervision
• Harnessing Evolution of Multi-Turn Conversations for Effective Answer Retrieval
• Technical report on Conversational Question Answering
• TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
• NewsPod: Automatic and Interactive News Podcasts
• Pirá: A Bilingual Portuguese-English Dataset for Question-Answering about the Ocean
• MG-ShopDial: A Multi-Goal Conversational Dataset for e-Commerce
• Hierarchical Context Tagging for Utterance Rewriting
• Report from the NSF Future Directions Workshop, Toward User-Oriented Agents: Research Directions and Challenges
• Natural Language QA Approaches using Reasoning with External Knowledge
• Variational Learning for Unsupervised Knowledge Grounded Dialogs
• Zero-Shot Open-Book Question Answering