Improving Question Answering by Commonsense-Based Pre-Training

Abstract

Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge. We believe the main reason is the lack of commonsense \mbox{connections} between concepts. To remedy this, we provide a simple and effective method that leverages external commonsense knowledge base such as ConceptNet. We pre-train direct and indirect relational functions between concepts, and show that these pre-trained functions could be easily added to existing neural network models. Results show that incorporating commonsense-based function improves the baseline on three question answering tasks that require commonsense reasoning. Further analysis shows that our system \mbox{discovers} and leverages useful evidence from an external commonsense knowledge base, which is missing in existing neural network models and help derive the correct answer.

arXiv comment7 pages

References in corpus (14)

• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Neural Natural Language Inference Models Enhanced with External Knowledge
• Adversarial Examples for Evaluating Reading Comprehension Systems
• A Neural Knowledge Language Model
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• A Tutorial on Network Embeddings
• KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
• Dynamic Fusion Networks for Machine Reading Comprehension
• Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing
• Answering Complex Questions Using Open Information Extraction
• HFL-RC System at SemEval-2018 Task 11: Hybrid Multi-Aspects Model for Commonsense Reading Comprehension
• Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension
• Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering

Cited by in corpus (8)

• ERNIE: Enhanced Language Representation with Informative Entities
• Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models
• Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach
• Neuro-symbolic Architectures for Context Understanding
• Answering Science Exam Questions Using Query Rewriting with Background Knowledge
• Knowledge Enhanced Attention for Robust Natural Language Inference
• Incorporating External Knowledge into Machine Reading for Generative Question Answering
• Beyond Language: Learning Commonsense from Images for Reasoning