Modeling Attention Flow on Graphs
arXiv:1811.00497
Abstract
Real-world scenarios demand reasoning about process, more than final outcome prediction, to discover latent causal chains and better understand complex systems. It requires the learning algorithms to offer both accurate predictions and clear interpretations. We design a set of trajectory reasoning tasks on graphs with only the source and the destination observed. We present the attention flow mechanism to explicitly model the reasoning process, leveraging the relational inductive biases by basing our models on graph networks. We study the way attention flow can effectively act on the underlying information flow implemented by message passing. Experiments demonstrate that the attention flow driven by and interacting with graph networks can provide higher accuracy in prediction and better interpretation for trajectory reasoning.
arXiv commentNeurIPS 2018 Relational Representation Learning Workshop
References in corpus (18)
- Neural Machine Translation by Jointly Learning to Align and Translate
- Semi-Supervised Classification with Graph Convolutional Networks
- Inductive Representation Learning on Large Graphs
- Neural Message Passing for Quantum Chemistry
- Geometric deep learning: going beyond Euclidean data
- Categorical Reparameterization with Gumbel-Softmax
- Spectral Networks and Locally Connected Networks on Graphs
- Deep Convolutional Networks on Graph-Structured Data
- Learning Combinatorial Optimization Algorithms over Graphs
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Interaction Networks for Learning about Objects, Relations and Physics
- Gated Graph Sequence Neural Networks
- Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
- Graph networks as learnable physics engines for inference and control
- TensorLog: A Differentiable Deductive Database
- Discovering objects and their relations from entangled scene representations
- Relational inductive bias for physical construction in humans and machines
- TensorLog: Deep Learning Meets Probabilistic DBs