Dynamic Coattention Networks For Question Answering

Abstract

Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus on relevant parts of both. Then a dynamic pointing decoder iterates over potential answer spans. This iterative procedure enables the model to recover from initial local maxima corresponding to incorrect answers. On the Stanford question answering dataset, a single DCN model improves the previous state of the art from 71.0% F1 to 75.9%, while a DCN ensemble obtains 80.4% F1.

14 pages, 7 figures, International Conference on Learning Representations 2017

References in corpus (6)

• Hierarchical Question-Image Co-Attention for Visual Question Answering
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Pointer Sentinel Mixture Models
• Machine Comprehension Using Match-LSTM and Answer Pointer
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension

Cited by in corpus (112)

• Deep Learning based Recommender System: A Survey and New Perspectives
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Deep Reinforcement Learning: An Overview
• Attention is not Explanation
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• The Natural Language Decathlon: Multitask Learning as Question Answering
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Natural Language Processing Advancements By Deep Learning: A Survey
• Biomedical Question Answering: A Survey of Approaches and Challenges
• Multi-Perspective Context Matching for Machine Comprehension
• Inf-VAE: A Variational Autoencoder Framework to Integrate Homophily and Influence in Diffusion Prediction
• Reading Wikipedia to Answer Open-Domain Questions
• Paradigm Shift in Natural Language Processing
• Spatio-Temporal Self-Attention Network for Video Saliency Prediction
• Global-Locally Self-Attentive Dialogue State Tracker
• Graph Neural Networks for Natural Language Processing: A Survey
• MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
• Learning to Compute Word Embeddings On the Fly
• A Survey on Transfer Learning in Natural Language Processing
• A Unified MRC Framework for Named Entity Recognition
• DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
• A Unified Query-based Generative Model for Question Generation and Question Answering
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
• Cross-Lingual Machine Reading Comprehension
• BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering
• Making Neural QA as Simple as Possible but not Simpler
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Multi-hop Question Generation with Graph Convolutional Network
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• U-Net: Machine Reading Comprehension with Unanswerable Questions
• A Survey of Natural Language Generation Techniques with a Focus on Dialogue Systems - Past, Present and Future Directions
• Stochastic Answer Networks for Machine Reading Comprehension
• Multi-range Reasoning for Machine Comprehension
• Description Based Text Classification with Reinforcement Learning
• Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions
• A Comparative Study of Word Embeddings for Reading Comprehension
• Machine Reading Comprehension: a Literature Review
• Semi-Supervised QA with Generative Domain-Adaptive Nets
• Neural Machine Reading Comprehension: Methods and Trends
• Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
• Unifying Question Answering, Text Classification, and Regression via Span Extraction
• Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks
• MemexQA: Visual Memex Question Answering
• Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction
• Survey of Visual Question Answering: Datasets and Techniques
• Dynamic Fusion Networks for Machine Reading Comprehension
• Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents
• MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data
• Reasoning with Sarcasm by Reading In-between
• Question Answering from Unstructured Text by Retrieval and Comprehension
• Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
• Structural Embedding of Syntactic Trees for Machine Comprehension
• Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
• Span-based Localizing Network for Natural Language Video Localization
• Smarnet: Teaching Machines to Read and Comprehend Like Human
• Question Answering over Knowledge Base using Language Model Embeddings
• Ruminating Reader: Reasoning with Gated Multi-Hop Attention
• Implicit Motion-Compensated Network for Unsupervised Video Object Segmentation
• Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering
• MCQA: Multimodal Co-attention Based Network for Question Answering
• A Comprehensive Analysis of Static Word Embeddings for Turkish
• Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
• Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• Deep Co-attention based Comparators For Relative Representation Learning in Person Re-identification
• Adversarial Cross-Domain Action Recognition with Co-Attention
• Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension
• Hierarchical Question Answering for Long Documents
• Cross-Lingual Transfer Learning for Question Answering
• RefNet: A Reference-aware Network for Background Based Conversation
• Adversarial Domain Adaptation for Machine Reading Comprehension
• HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering
• Conversations with Search Engines: SERP-based Conversational Response Generation
• DLGNet: A Transformer-based Model for Dialogue Response Generation
• Around the GLOBE: Numerical Aggregation Question-Answering on Heterogeneous Genealogical Knowledge Graphs with Deep Neural Networks
• Learning to Search in Long Documents Using Document Structure
• Graph Sequential Network for Reasoning over Sequences
• Learning Context-Sensitive Convolutional Filters for Text Processing
• Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
• Reinforced Dynamic Reasoning for Conversational Question Generation
• Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
• Multi-Mention Learning for Reading Comprehension with Neural Cascades
• Self-Attentive Neural Collaborative Filtering
• Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task
• Tackling Graphical NLP problems with Graph Recurrent Networks
• How to Become Instagram Famous: Post Popularity Prediction with Dual-Attention
• Refining Raw Sentence Representations for Textual Entailment Recognition via Attention
• A Boo(n) for Evaluating Architecture Performance
• Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation
• Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks
• Implicit Argument Prediction as Reading Comprehension
• Query-Based Named Entity Recognition
• Deep Human Answer Understanding for Natural Reverse QA
• Relational dynamic memory networks
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation
• Making the Best Use of Review Summary for Sentiment Analysis
• An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks
• A Study of the Tasks and Models in Machine Reading Comprehension
• Pay More Attention - Neural Architectures for Question-Answering
• Learning to Generate Questions with Adaptive Copying Neural Networks
• The emergent algebraic structure of RNNs and embeddings in NLP
• CAESAR: Context Awareness Enabled Summary-Attentive Reader
• Dual Attention Network for Product Compatibility and Function Satisfiability Analysis
• Question-Aware Sentence Gating Networks for Question and Answering
• DCA: Diversified Co-Attention towards Informative Live Video Commenting
• Learning to Generate Structured Queries from Natural Language with Indirect Supervision
• Mapping Natural Language Commands to Web Elements
• Bi-directional Cognitive Thinking Network for Machine Reading Comprehension
• A Coarse to Fine Question Answering System based on Reinforcement Learning
• Retrieving and ranking short medical questions with two stages neural matching model