Neural Consciousness Flow

Abstract

The ability of reasoning beyond data fitting is substantial to deep learning systems in order to make a leap forward towards artificial general intelligence. A lot of efforts have been made to model neural-based reasoning as an iterative decision-making process based on recurrent networks and reinforcement learning. Instead, inspired by the consciousness prior proposed by Yoshua Bengio, we explore reasoning with the notion of attentive awareness from a cognitive perspective, and formulate it in the form of attentive message passing on graphs, called neural consciousness flow (NeuCFlow). Aiming to bridge the gap between deep learning systems and reasoning, we propose an attentive computation framework with a three-layer architecture, which consists of an unconsciousness flow layer, a consciousness flow layer, and an attention flow layer. We implement the NeuCFlow model with graph neural networks (GNNs) and conditional transition matrices. Our attentive computation greatly reduces the complexity of vanilla GNN-based methods, capable of running on large-scale graphs. We validate our model for knowledge graph reasoning by solving a series of knowledge base completion (KBC) tasks. The experimental results show NeuCFlow significantly outperforms previous state-of-the-art KBC methods, including the embedding-based and the path-based. The reproducible code can be found by the link below.

References in corpus (23)

• Semi-Supervised Classification with Graph Convolutional Networks
• Geometric deep learning: going beyond Euclidean data
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• A Structured Self-attentive Sentence Embedding
• Deep Convolutional Networks on Graph-Structured Data
• A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network
• A simple neural network module for relational reasoning
• MolGAN: An implicit generative model for small molecular graphs
• Learning Deep Generative Models of Graphs
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• Neural Combinatorial Optimization with Reinforcement Learning
• Graph networks as learnable physics engines for inference and control
• Learning a SAT Solver from Single-Bit Supervision
• A Compositional Object-Based Approach to Learning Physical Dynamics
• Relational Deep Reinforcement Learning
• The Consciousness Prior
• Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach
• TensorLog: A Differentiable Deductive Database
• Discovering objects and their relations from entangled scene representations
• M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search
• Metacontrol for Adaptive Imagination-Based Optimization
• Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
• Action Schema Networks: Generalised Policies with Deep Learning