Making Neural QA as Simple as Possible but not Simpler

Abstract

Recent development of large-scale question answering (QA) datasets triggered a substantial amount of research into end-to-end neural architectures for QA. Increasingly complex systems have been conceived without comparison to simpler neural baseline systems that would justify their complexity. In this work, we propose a simple heuristic that guides the development of neural baseline systems for the extractive QA task. We find that there are two ingredients necessary for building a high-performing neural QA system: first, the awareness of question words while processing the context and second, a composition function that goes beyond simple bag-of-words modeling, such as recurrent neural networks. Our results show that FastQA, a system that meets these two requirements, can achieve very competitive performance compared with existing models. We argue that this surprising finding puts results of previous systems and the complexity of recent QA datasets into perspective.

References in corpus (7)

• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• Dynamic Coattention Networks For Question Answering
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Multi-Perspective Context Matching for Machine Comprehension
• NewsQA: A Machine Comprehension Dataset
• Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering
• Words or Characters? Fine-grained Gating for Reading Comprehension

Cited by in corpus (28)

• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Adversarial Examples for Evaluating Reading Comprehension Systems
• DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
• Dynamic Integration of Background Knowledge in Neural NLU Systems
• Multilingual Extractive Reading Comprehension by Runtime Machine Translation
• S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
• The price of debiasing automatic metrics in natural language evaluation
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Stochastic Answer Networks for Machine Reading Comprehension
• Machine Reading Comprehension: a Literature Review
• Efficient and Robust Question Answering from Minimal Context over Documents
• Focused Hierarchical RNNs for Conditional Sequence Processing
• A Survey on Neural Machine Reading Comprehension
• Adversarial Text Generation Without Reinforcement Learning
• Dynamic Fusion Networks for Machine Reading Comprehension
• What Makes Reading Comprehension Questions Easier?
• Ruminating Reader: Reasoning with Gated Multi-Hop Attention
• Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
• HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering
• Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension
• ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
• NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language
• A Study of the Tasks and Models in Machine Reading Comprehension
• An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks
• Translating Questions into Answers using DBPedia n-triples
• End-to-End Learning for Answering Structured Queries Directly over Text
• A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension