Neural Relational Inference for Interacting Systems

Abstract

Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics. The interplay of components can give rise to complex behavior, which can often be explained using a simple model of the system's constituent parts. In this work, we introduce the neural relational inference (NRI) model: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data. Our model takes the form of a variational auto-encoder, in which the latent code represents the underlying interaction graph and the reconstruction is based on graph neural networks. In experiments on simulated physical systems, we show that our NRI model can accurately recover ground-truth interactions in an unsupervised manner. We further demonstrate that we can find an interpretable structure and predict complex dynamics in real motion capture and sports tracking data.

arXiv commentICML (2018). Code available under https://github.com/ethanfetaya/NRI

References in corpus (15)

• Auto-Encoding Variational Bayes
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Categorical Reparameterization with Gumbel-Softmax
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• A Structured Self-attentive Sentence Embedding
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Interaction Networks for Learning about Objects, Relations and Physics
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Few-Shot Learning with Graph Neural Networks
• Variational Lossy Autoencoder
• A Compositional Object-Based Approach to Learning Physical Dynamics
• Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
• Permutation-equivariant neural networks applied to dynamics prediction

Cited by in corpus (62)

• Deep Learning for Time Series Forecasting: Tutorial and Literature Survey
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Discovering Symbolic Models from Deep Learning with Inductive Biases
• Learning Discrete Structures for Graph Neural Networks
• Flexible Neural Representation for Physics Prediction
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• E(n) Equivariant Graph Neural Networks
• Representation Learning for Natural Language Processing
• NF-GNN: Network Flow Graph Neural Networks for Malware Detection and Classification
• K-Core based Temporal Graph Convolutional Network for Dynamic Graphs
• Graph Neural Ordinary Differential Equations
• Thermodynamics-informed graph neural networks
• Physics-Informed Supervised Residual Learning for Electromagnetic Modeling
• Learning Dynamic Belief Graphs to Generalize on Text-Based Games
• Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
• Hyperbolic Attention Networks
• Indirect Adversarial Attacks via Poisoning Neighbors for Graph Convolutional Networks
• Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detection
• Learning Symbolic Physics with Graph Networks
• Edge-labeling Graph Neural Network for Few-shot Learning
• Discrete and continuous representations and processing in deep learning: Looking forward
• DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks
• Finding Patient Zero: Learning Contagion Source with Graph Neural Networks
• Learning Continuous System Dynamics from Irregularly-Sampled Partial Observations
• Operationally meaningful representations of physical systems in neural networks
• Learning Sparse Interaction Graphs of Partially Detected Pedestrians for Trajectory Prediction
• Kernel-based Graph Learning from Smooth Signals: A Functional Viewpoint
• Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity
• Pointer Graph Networks
• Graph Neural Networks to Predict Sports Outcomes
• Spatial Dual-Modality Graph Reasoning for Key Information Extraction
• Set2Graph: Learning Graphs From Sets
• Named Entity Disambiguation using Deep Learning on Graphs
• Deep Imitation Learning for Bimanual Robotic Manipulation
• Discrete-Valued Neural Communication
• Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling
• Focusing and Diffusion: Bidirectional Attentive Graph Convolutional Networks for Skeleton-based Action Recognition
• Flow-based Spatio-Temporal Structured Prediction of Motion Dynamics
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction
• Temporal Attribute Prediction via Joint Modeling of Multi-Relational Structure Evolution
• Learning Tree Structures from Leaves For Particle Decay Reconstruction
• Disentangled Neural Relational Inference for Interpretable Motion Prediction
• Compositional Video Synthesis with Action Graphs
• SENSORIMOTOR GRAPH: Action-Conditioned Graph Neural Network for Learning Robotic Soft Hand Dynamics
• Inference for Network Structure and Dynamics from Time Series Data via Graph Neural Network
• Accurately Solving Physical Systems with Graph Learning
• Graph Neural Networks Including Sparse Interpretability
• Deep learning reveals hidden interactions in complex systems
• Alternating ConvLSTM: Learning Force Propagation with Alternate State Updates
• Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
• Scan2Mesh: From Unstructured Range Scans to 3D Meshes
• Improved Structural Discovery and Representation Learning of Multi-Agent Data
• Modular meta-learning in abstract graph networks for combinatorial generalization
• Deep Dynamic Effective Connectivity Estimation from Multivariate Time Series
• Relate and Predict: Structure-Aware Prediction with Jointly Optimized Neural DAG
• Learning to Extend Program Graphs to Work-in-Progress Code
• Discovering Latent Representations of Relations for Interacting Systems
• Uncertainty in Neural Relational Inference Trajectory Reconstruction
• Policy Message Passing: A New Algorithm for Probabilistic Graph Inference