Permutation-equivariant neural networks applied to dynamics prediction

Abstract

The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which there are a number of different inputs which are all 'of the same type' --- multiple particles, multiple agents, multiple stock prices, etc. The corresponding symmetry to this is permutation symmetry, in that the algorithm should not depend on the specific ordering of the input data. We discuss a permutation-invariant neural network layer in analogy to convolutional layers, and show the ability of this architecture to learn to predict the motion of a variable number of interacting hard discs in 2D. In the same way that convolutional layers can generalize to different image sizes, the permutation layer we describe generalizes to different numbers of objects.

7 pages, 4 figures

Cited by in corpus (21)

• Neural Relational Inference for Interacting Systems
• Sampling using $SU(N)$ gauge equivariant flows
• Hopfield Networks is All You Need
• A Likelihood-Free Inference Framework for Population Genetic Data using Exchangeable Neural Networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning
• Discovering Nonlinear Relations with Minimum Predictive Information Regularization
• Factorised Neural Relational Inference for Multi-Interaction Systems
• The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
• Neural Clustering Processes
• The general theory of permutation equivarant neural networks and higher order graph variational encoders
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Circular-Symmetric Correlation Layer based on FFT
• Geared Rotationally Identical and Invariant Convolutional Neural Network Systems
• Equivariant Deep Dynamical Model for Motion Prediction
• Correlation-aware Unsupervised Change-point Detection via Graph Neural Networks
• Transformationally Identical and Invariant Convolutional Neural Networks through Symmetric Element Operators
• Link Prediction on N-ary Relational Data Based on Relatedness Evaluation
• SetGAN: Improving the stability and diversity of generative models through a permutation invariant architecture
• Set Cross Entropy: Likelihood-based Permutation Invariant Loss Function for Probability Distributions