SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering

Abstract

Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context. Different from traditional single-turn machine reading comprehension (MRC) tasks, CQA includes passage comprehension, coreference resolution, and contextual understanding. In this paper, we propose an innovated contextualized attention-based deep neural network, SDNet, to fuse context into traditional MRC models. Our model leverages both inter-attention and self-attention to comprehend conversation context and extract relevant information from passage. Furthermore, we demonstrated a novel method to integrate the latest BERT contextual model. Empirical results show the effectiveness of our model, which sets the new state of the art result in CoQA leaderboard, outperforming the previous best model by 1.6% F1. Our ensemble model further improves the result by 2.7% F1.

8 pages, 2 figures

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Variational Dropout and the Local Reparameterization Trick
• Reading Wikipedia to Answer Open-Domain Questions
• QuAC : Question Answering in Context
• A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC

Cited by in corpus (34)

• GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
• Open-Retrieval Conversational Question Answering
• Interactive Question Answering Systems: Literature Review
• Technical report on Conversational Question Answering
• Neural Machine Reading Comprehension: Methods and Trends
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering
• GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
• Dialog State Tracking: A Neural Reading Comprehension Approach
• Review Conversational Reading Comprehension
• TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
• MA-DST: Multi-Attention Based Scalable Dialog State Tracking
• FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension
• A Survey of Document Grounded Dialogue Systems (DGDS)
• Mixed-Lingual Pre-training for Cross-lingual Summarization
• Leveraging Lead Bias for Zero-shot Abstractive News Summarization
• A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction
• Deepening Hidden Representations from Pre-trained Language Models
• Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension
• Reinforced Dynamic Reasoning for Conversational Question Generation
• ChainCQG: Flow-Aware Conversational Question Generation
• Conversational Question Answering: A Survey
• Structural Pre-training for Dialogue Comprehension
• A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension
• An Empirical Study of Content Understanding in Conversational Question Answering
• Weakly-Supervised Open-Retrieval Conversational Question Answering
• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
• Towards Adaptive Context Management for Intelligent Conversational Question Answering
• Knowledge Distillation for Improved Accuracy in Spoken Question Answering
• Hierarchical Multitask Learning Approach for BERT
• Data Augmentation for Spoken Language Understanding via Pretrained Language Models
• Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering
• MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
• Opinion Prediction with User Fingerprinting
• Self-supervised Dialogue Learning for Spoken Conversational Question Answering