Learning Contextualized Knowledge Structures for Commonsense Reasoning
arXiv:2010.12873
Abstract
Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks. However, KG edge (fact) sparsity and noisy edge extraction/generation often hinder models from obtaining useful knowledge to reason over. To address these issues, we propose a new KG-augmented model: Hybrid Graph Network (HGN). Unlike prior methods, HGN learns to jointly contextualize extracted and generated knowledge by reasoning over both within a unified graph structure. Given the task input context and an extracted KG subgraph, HGN is trained to generate embeddings for the subgraph's missing edges to form a "hybrid" graph, then reason over the hybrid graph while filtering out context-irrelevant edges. We demonstrate HGN's effectiveness through considerable performance gains across four commonsense reasoning benchmarks, plus a user study on edge validness and helpfulness.
arXiv commentAccepted to Findings of ACL-IJCNLP 2021. Code and data: https://github.com/INK-USC/HGN
References in corpus (10)
- ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
- Deep Learning: A Critical Appraisal
- Learning Discrete Structures for Graph Neural Networks
- Learning Conditioned Graph Structures for Interpretable Visual Question Answering
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
- Machine Common Sense Concept Paper
- Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering
- KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
- Fusing Context Into Knowledge Graph for Commonsense Question Answering
- Careful Selection of Knowledge to solve Open Book Question Answering