A simple neural network module for relational reasoning
arXiv:1706.01427
Abstract
Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn. In this paper we describe how to use Relation Networks (RNs) as a simple plug-and-play module to solve problems that fundamentally hinge on relational reasoning. We tested RN-augmented networks on three tasks: visual question answering using a challenging dataset called CLEVR, on which we achieve state-of-the-art, super-human performance; text-based question answering using the bAbI suite of tasks; and complex reasoning about dynamic physical systems. Then, using a curated dataset called Sort-of-CLEVR we show that powerful convolutional networks do not have a general capacity to solve relational questions, but can gain this capacity when augmented with RNs. Our work shows how a deep learning architecture equipped with an RN module can implicitly discover and learn to reason about entities and their relations.
References in corpus (5)
- Semi-Supervised Classification with Graph Convolutional Networks
- Deep Convolutional Networks on Graph-Structured Data
- Towards Deep Symbolic Reinforcement Learning
- Discovering objects and their relations from entangled scene representations
- A Pooling Approach to Modelling Spatial Relations for Image Retrieval and Annotation
Cited by in corpus (7)
- Attentional Pooling for Action Recognition
- FiLM: Visual Reasoning with a General Conditioning Layer
- GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition
- A Simple Loss Function for Improving the Convergence and Accuracy of Visual Question Answering Models
- Predicting the Popularity of Online Videos via Deep Neural Networks
- Grounded Objects and Interactions for Video Captioning
- Dynamic Input Structure and Network Assembly for Few-Shot Learning