Interaction Networks for Learning about Objects, Relations and Physics

Abstract

Reasoning about objects, relations, and physics is central to human intelligence, and a key goal of artificial intelligence. Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system. Our model takes graphs as input, performs object- and relation-centric reasoning in a way that is analogous to a simulation, and is implemented using deep neural networks. We evaluate its ability to reason about several challenging physical domains: n-body problems, rigid-body collision, and non-rigid dynamics. Our results show it can be trained to accurately simulate the physical trajectories of dozens of objects over thousands of time steps, estimate abstract quantities such as energy, and generalize automatically to systems with different numbers and configurations of objects and relations. Our interaction network implementation is the first general-purpose, learnable physics engine, and a powerful general framework for reasoning about object and relations in a wide variety of complex real-world domains.

Published in NIPS 2016

References in corpus (1)

• Deep Learning in Neural Networks: An Overview

Cited by in corpus (17)

• Covariant Compositional Networks For Learning Graphs
• Explainability Techniques for Graph Convolutional Networks
• Situation-Aware Pedestrian Trajectory Prediction with Spatio-Temporal Attention Model
• Learning A Physical Long-term Predictor
• Spatial-Temporal Relation Networks for Multi-Object Tracking
• Reasoning Visual Dialogs with Structural and Partial Observations
• Learning Reciprocity in Complex Sequential Social Dilemmas
• A Perspective on Objects and Systematic Generalization in Model-Based RL
• AliGraph: A Comprehensive Graph Neural Network Platform
• Factor Graph Neural Network
• Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces
• Intrinsic Motivation Driven Intuitive Physics Learning using Deep Reinforcement Learning with Intrinsic Reward Normalization
• Visual Semantic Information Pursuit: A Survey
• Unsupervised Intuitive Physics from Past Experiences
• Structured agents for physical construction
• Typed Graph Networks
• Human-Guided Learning of Column Networks: Augmenting Deep Learning with Advice