Relational dynamic memory networks
arXiv:1808.04247
Abstract
Neural networks excel in detecting regular patterns but are less successful in representing and manipulating complex data structures, possibly due to the lack of an external memory. This has led to the recent development of a new line of architectures known as Memory-Augmented Neural Networks (MANNs), each of which consists of a neural network that interacts with an external memory matrix. However, this RAM-like memory matrix is unstructured and thus does not naturally encode structured objects. Here we design a new MANN dubbed Relational Dynamic Memory Network (RMDN) to bridge the gap. Like existing MANNs, RMDN has a neural controller but its memory is structured as multi-relational graphs. RMDN uses the memory to represent and manipulate graph-structured data in response to query; and as a neural network, RMDN is trainable from labeled data. Thus RMDN learns to answer queries about a set of graph-structured objects without explicit programming. We evaluate the capability of RMDN on several important prediction problems, including software vulnerability, molecular bioactivity and chemical-chemical interaction. Results demonstrate the efficacy of the proposed model.
arXiv commentPrevious versions published in 3rd Representation Learning for Graphs Workshop (ReLiG 2017), ICPR'18, and NeurIPS'18 Workshop on machine learning for molecules and materials; arXiv admin note: text overlap with arXiv:1801.02622"
References in corpus (24)
- Neural Message Passing for Quantum Chemistry
- Geometric deep learning: going beyond Euclidean data
- Deep Convolutional Networks on Graph-Structured Data
- Training Very Deep Networks
- Hierarchical Graph Representation Learning with Differentiable Pooling
- Learning Convolutional Neural Networks for Graphs
- End-To-End Memory Networks
- A simple neural network module for relational reasoning
- Dynamic Coattention Networks For Question Answering
- GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models
- Learning Deep Generative Models of Graphs
- Massively Multitask Networks for Drug Discovery
- Tracking the World State with Recurrent Entity Networks
- Multi-task Neural Networks for QSAR Predictions
- A deep tree-based model for software defect prediction
- Highway and Residual Networks learn Unrolled Iterative Estimation
- Toxicity Prediction using Deep Learning
- Automatic feature learning for vulnerability prediction
- Modeling Relational Data with Graph Convolutional Networks
- Deep Models of Interactions Across Sets
- Relational recurrent neural networks
- DeepCCI: End-to-end Deep Learning for Chemical-Chemical Interaction Prediction
- Learning Deep Matrix Representations
- Attentional Multilabel Learning over Graphs: A Message Passing Approach