GNN is a Counter? Revisiting GNN for Question Answering

Abstract

Question Answering (QA) has been a long-standing research topic in AI and NLP fields, and a wealth of studies have been conducted to attempt to equip QA systems with human-level reasoning capability. To approximate the complicated human reasoning process, state-of-the-art QA systems commonly use pre-trained language models (LMs) to access knowledge encoded in LMs together with elaborately designed modules based on Graph Neural Networks (GNNs) to perform reasoning over knowledge graphs (KGs). However, many problems remain open regarding the reasoning functionality of these GNN-based modules. Can these GNN-based modules really perform a complex reasoning process? Are they under- or over-complicated for QA? To open the black box of GNN and investigate these problems, we dissect state-of-the-art GNN modules for QA and analyze their reasoning capability. We discover that even a very simple graph neural counter can outperform all the existing GNN modules on CommonsenseQA and OpenBookQA, two popular QA benchmark datasets which heavily rely on knowledge-aware reasoning. Our work reveals that existing knowledge-aware GNN modules may only carry out some simple reasoning such as counting. It remains a challenging open problem to build comprehensive reasoning modules for knowledge-powered QA.

References in corpus (21)

• Semi-Supervised Classification with Graph Convolutional Networks
• Language Models are Few-Shot Learners
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• End-To-End Memory Networks
• A simple neural network module for relational reasoning
• Variational Dropout and the Local Reparameterization Trick
• Variational Dropout Sparsifies Deep Neural Networks
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• Learning Efficient Convolutional Networks through Network Slimming
• Variational Reasoning for Question Answering with Knowledge Graph
• Language Models as Knowledge Bases?
• HAQ: Hardware-Aware Automated Quantization with Mixed Precision
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
• Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge
• Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
• From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
• Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering
• Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering