A Compositional Object-Based Approach to Learning Physical Dynamics

Abstract

We present the Neural Physics Engine (NPE), a framework for learning simulators of intuitive physics that naturally generalize across variable object count and different scene configurations. We propose a factorization of a physical scene into composable object-based representations and a neural network architecture whose compositional structure factorizes object dynamics into pairwise interactions. Like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions; realized as a neural network, it can be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that the NPE's compositional representation of the structure in physical interactions improves its ability to predict movement, generalize across variable object count and different scene configurations, and infer latent properties of objects such as mass.

Published as a conference paper for ICLR 2017. 15 pages, 6 figures

Cited by in corpus (36)

• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Hamiltonian Graph Networks with ODE Integrators
• Explainability Techniques for Graph Convolutional Networks
• On the Binding Problem in Artificial Neural Networks
• Differentiable Physics-informed Graph Networks
• gradSim: Differentiable simulation for system identification and visuomotor control
• Reasoning About Physical Interactions with Object-Oriented Prediction and Planning
• Learning A Physical Long-term Predictor
• Learning Continuous System Dynamics from Irregularly-Sampled Partial Observations
• Learning Robotic Manipulation of Granular Media
• Multi-agent Trajectory Prediction with Fuzzy Query Attention
• Structured Object-Aware Physics Prediction for Video Modeling and Planning
• PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics
• Active Learning of Abstract Plan Feasibility
• 3D-OES: Viewpoint-Invariant Object-Factorized Environment Simulators
• Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces
• Occlusion resistant learning of intuitive physics from videos
• Object-centric Forward Modeling for Model Predictive Control
• Active World Model Learning with Progress Curiosity
• Programmable Agents
• Learning Generalizable Physical Dynamics of 3D Rigid Objects
• Simulating Continuum Mechanics with Multi-Scale Graph Neural Networks
• Unsupervised Intuitive Physics from Past Experiences
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos
• Efficient Cloth Simulation using Miniature Cloth and Upscaling Deep Neural Networks
• RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks
• Unsupervised Object Keypoint Learning using Local Spatial Predictability
• Relational State-Space Model for Stochastic Multi-Object Systems
• Structured agents for physical construction
• Causal World Models by Unsupervised Deconfounding of Physical Dynamics
• Learning Representation over Dynamic Graph using Aggregation-Diffusion Mechanism
• Adding Intuitive Physics to Neural-Symbolic Capsules Using Interaction Networks
• Neural Consciousness Flow
• Predicting the Physical Dynamics of Unseen 3D Objects
• cvpaper.challenge in 2016: Futuristic Computer Vision through 1,600 Papers Survey