Discovering objects and their relations from entangled scene representations

Abstract

Our world can be succinctly and compactly described as structured scenes of objects and relations. A typical room, for example, contains salient objects such as tables, chairs and books, and these objects typically relate to each other by their underlying causes and semantics. This gives rise to correlated features, such as position, function and shape. Humans exploit knowledge of objects and their relations for learning a wide spectrum of tasks, and more generally when learning the structure underlying observed data. In this work, we introduce relation networks (RNs) - a general purpose neural network architecture for object-relation reasoning. We show that RNs are capable of learning object relations from scene description data. Furthermore, we show that RNs can act as a bottleneck that induces the factorization of objects from entangled scene description inputs, and from distributed deep representations of scene images provided by a variational autoencoder. The model can also be used in conjunction with differentiable memory mechanisms for implicit relation discovery in one-shot learning tasks. Our results suggest that relation networks are a potentially powerful architecture for solving a variety of problems that require object relation reasoning.

ICLR Workshop 2017

Cited by in corpus (37)

• Graph Neural Networks: A Review of Methods and Applications
• A simple neural network module for relational reasoning
• Discovering physical concepts with neural networks
• Graph networks as learnable physics engines for inference and control
• Pixels to Graphs by Associative Embedding
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Deformable ConvNets v2: More Deformable, Better Results
• Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks
• Are Disentangled Representations Helpful for Abstract Visual Reasoning?
• Symbol Emergence in Cognitive Developmental Systems: a Survey
• Recurrent Independent Mechanisms
• Relation Networks for Object Detection
• Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
• ShapeWorld - A new test methodology for multimodal language understanding
• Self-Supervised Relational Reasoning for Representation Learning
• Graph Neural Based End-to-end Data Association Framework for Online Multiple-Object Tracking
• Relational recurrent neural networks
• Discrete-Valued Neural Communication
• The Limited Multi-Label Projection Layer
• Learning Physical Constraints with Neural Projections
• Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects
• Fashion Recommendation and Compatibility Prediction Using Relational Network
• Attention over learned object embeddings enables complex visual reasoning
• Are Graph Neural Networks Miscalibrated?
• Modeling Attention Flow on Graphs
• Do Vision Models Encode Object-Level Semantic Relatedness? A Cognitive Psychology-Inspired Benchmark
• Scaling Graph-based Deep Learning models to larger networks
• Graph Neural Processes: Towards Bayesian Graph Neural Networks
• Topos and Stacks of Deep Neural Networks
• Disentangled Relational Representations for Explaining and Learning from Demonstration
• Typed Graph Networks
• Image-Level Attentional Context Modeling Using Nested-Graph Neural Networks
• Time-Series Event Prediction with Evolutionary State Graph
• Adding Intuitive Physics to Neural-Symbolic Capsules Using Interaction Networks
• Relational Reasoning Network (RRN) for Anatomical Landmarking
• Neural Consciousness Flow
• Anatomical labeling of brain CT scan anomalies using multi-context nearest neighbor relation networks