Dynamic Integration of Background Knowledge in Neural NLU Systems

Abstract

Common-sense and background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, this knowledge must be acquired from training corpora during learning, and then it is static at test time. We introduce a new architecture for the dynamic integration of explicit background knowledge in NLU models. A general-purpose reading module reads background knowledge in the form of free-text statements (together with task-specific text inputs) and yields refined word representations to a task-specific NLU architecture that reprocesses the task inputs with these representations. Experiments on document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of the approach. Analysis shows that our model learns to exploit knowledge in a semantically appropriate way.

References in corpus (6)

• Learned in Translation: Contextualized Word Vectors
• Natural Language Inference over Interaction Space
• Neural Models for Information Retrieval
• Linguistic Knowledge as Memory for Recurrent Neural Networks
• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
• A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse

Cited by in corpus (25)

• Unsupervised Question Answering by Cloze Translation
• Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
• Learning to Compute Word Embeddings On the Fly
• Machine Common Sense Concept Paper
• KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning
• Efficient and Robust Question Answering from Minimal Context over Documents
• Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
• Contextualized Representations Using Textual Encyclopedic Knowledge
• Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension
• Commonsense for Generative Multi-Hop Question Answering Tasks
• Why Do Masked Neural Language Models Still Need Common Sense Knowledge?
• Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• Reasoning Over Virtual Knowledge Bases With Open Predicate Relations
• Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs
• Bilinear Fusion of Commonsense Knowledge with Attention-Based NLI Models
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
• Incorporating External Knowledge into Machine Reading for Generative Question Answering
• CO-NNECT: A Framework for Revealing Commonsense Knowledge Paths as Explicitations of Implicit Knowledge in Texts
• Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
• ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation
• NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language
• A Study of the Tasks and Models in Machine Reading Comprehension
• What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering
• Understanding in Artificial Intelligence