Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing

Abstract

Machine Learning has been the quintessential solution for many AI problems, but learning is still heavily dependent on the specific training data. Some learning models can be incorporated with a prior knowledge in the Bayesian set up, but these learning models do not have the ability to access any organised world knowledge on demand. In this work, we propose to enhance learning models with world knowledge in the form of Knowledge Graph (KG) fact triples for Natural Language Processing (NLP) tasks. Our aim is to develop a deep learning model that can extract relevant prior support facts from knowledge graphs depending on the task using attention mechanism. We introduce a convolution-based model for learning representations of knowledge graph entity and relation clusters in order to reduce the attention space. We show that the proposed method is highly scalable to the amount of prior information that has to be processed and can be applied to any generic NLP task. Using this method we show significant improvement in performance for text classification with News20, DBPedia datasets and natural language inference with Stanford Natural Language Inference (SNLI) dataset. We also demonstrate that a deep learning model can be trained well with substantially less amount of labeled training data, when it has access to organised world knowledge in the form of knowledge graph.

arXiv commentAccepted at NAACL 2018

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• LSTM: A Search Space Odyssey
• Convolutional Neural Networks for Sentence Classification
• A large annotated corpus for learning natural language inference
• ProjE: Embedding Projection for Knowledge Graph Completion
• SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions
• TransG : A Generative Mixture Model for Knowledge Graph Embedding
• From One Point to A Manifold: Knowledge Graph Embedding For Precise Link Prediction
• A Survey of Inductive Biases for Factorial Representation-Learning
• The More You Know: Using Knowledge Graphs for Image Classification
• Machine Learning with World Knowledge: The Position and Survey
• Deep Hashing: A Joint Approach for Image Signature Learning

Cited by in corpus (4)

• Improving Question Answering by Commonsense-Based Pre-Training
• CAVA: A Visual Analytics System for Exploratory Columnar Data Augmentation Using Knowledge Graphs
• GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search
• Cooperative Reasoning on Knowledge Graph and Corpus: A Multi-agentReinforcement Learning Approach