Universal approximations of permutation invariant/equivariant functions by deep neural networks

Abstract

In this paper, we develop a theory about the relationship between $G$-invariant/equivariant functions and deep neural networks for finite group $G$. Especially, for a given $G$-invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip $G$-actions and each affine transformations are $G$-equivariant/invariant. Due to representation theory, we can show that this approximator has exponentially fewer free parameters than usual models.

References in corpus (9)

• Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
• Group Equivariant Convolutional Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Approximating Continuous Functions by ReLU Nets of Minimal Width
• Universal Invariant and Equivariant Graph Neural Networks
• Invariant and Equivariant Graph Networks
• Covariant Compositional Networks For Learning Graphs
• Deep Models of Interactions Across Sets
• Probabilistic symmetries and invariant neural networks

Cited by in corpus (27)

• Machine Learning in QM/MM Molecular Dynamics Simulations of Condensed-Phase Systems
• Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions
• Universal Invariant and Equivariant Graph Neural Networks
• Sampling using $SU(N)$ gauge equivariant flows
• Are Transformers universal approximators of sequence-to-sequence functions?
• Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Recent advances in deep learning theory
• Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks
• Set2Graph: Learning Graphs From Sets
• Exchangeable deep neural networks for set-to-set matching and learning
• A Simple Proof of the Universality of Invariant/Equivariant Graph Neural Networks
• Fixed-Dimensional and Permutation Invariant State Representation of Autonomous Driving
• Universal Equivariant Multilayer Perceptrons
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Universal Approximation Theorem for Equivariant Maps by Group CNNs
• Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers
• On Universal Equivariant Set Networks
• Addressing imperfect symmetry: A novel symmetry-learning actor-critic extension
• Graph Homomorphism Convolution
• The general theory of permutation equivarant neural networks and higher order graph variational encoders
• Improved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces
• Set Aggregation Network as a Trainable Pooling Layer
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Neural Trees for Learning on Graphs
• Learning Set-equivariant Functions with SWARM Mappings
• Neural Networks on Groups