QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

Abstract

The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify relevant knowledge from large KGs, and (ii) perform joint reasoning over the QA context and KG. In this work, we propose a new model, QA-GNN, which addresses the above challenges through two key innovations: (i) relevance scoring, where we use LMs to estimate the importance of KG nodes relative to the given QA context, and (ii) joint reasoning, where we connect the QA context and KG to form a joint graph, and mutually update their representations through graph neural networks. We evaluate our model on QA benchmarks in the commonsense (CommonsenseQA, OpenBookQA) and biomedical (MedQA-USMLE) domains. QA-GNN outperforms existing LM and LM+KG models, and exhibits capabilities to perform interpretable and structured reasoning, e.g., correctly handling negation in questions.

NAACL 2021. Code & data available at https://github.com/michiyasunaga/qagnn

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• On the difficulty of training Recurrent Neural Networks
• Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
• Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
• Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering
• Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering
• Path-Based Contextualization of Knowledge Graphs for Textual Entailment

Cited by in corpus (11)

• On the Opportunities and Risks of Foundation Models
• Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction
• Graph Neural Networks for Natural Language Processing: A Survey
• Enhancing Heterogeneous Knowledge Graph Completion with a Novel GAT-based Approach
• Learning Representations of Bi-level Knowledge Graphs for Reasoning beyond Link Prediction
• GNN is a Counter? Revisiting GNN for Question Answering
• SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs
• Enhancing Low-Resource Relation Representations through Multi-View Decoupling
• Graph-augmented Learning to Rank for Querying Large-scale Knowledge Graph
• Zero-Shot Open-Book Question Answering
• SQALER: Scaling Question Answering by Decoupling Multi-Hop and Logical Reasoning