On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups

Abstract

Convolutional neural networks have been extremely successful in the image recognition domain because they ensure equivariance to translations. There have been many recent attempts to generalize this framework to other domains, including graphs and data lying on manifolds. In this paper we give a rigorous, theoretical treatment of convolution and equivariance in neural networks with respect to not just translations, but the action of any compact group. Our main result is to prove that (given some natural constraints) convolutional structure is not just a sufficient, but also a necessary condition for equivariance to the action of a compact group. Our exposition makes use of concepts from representation theory and noncommutative harmonic analysis and derives new generalized convolution formulae.

arXiv commentFinal version that appeared in the proceedings of the 35th International Conference on Machine Learning (ICML 2018), Stockholm, Sweden

References in corpus (8)

• Neural Message Passing for Quantum Chemistry
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Rotation equivariant vector field networks
• Equivariance Through Parameter-Sharing
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network

Cited by in corpus (113)

• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Data-driven rogue waves and parameter discovery in the defocusing NLS equation with a potential using the PINN deep learning
• Cormorant: Covariant Molecular Neural Networks
• Universal Invariant and Equivariant Graph Neural Networks
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
• 3D G-CNNs for Pulmonary Nodule Detection
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• PDE-based Group Equivariant Convolutional Neural Networks
• Simplicial Neural Networks
• On Learning Sets of Symmetric Elements
• Building spatial symmetries into parameterized quantum circuits for faster training
• Reflection Equivariant Quantum Neural Networks for Enhanced Image Classification
• Universal approximations of permutation invariant/equivariant functions by deep neural networks
• Scale-Equivariant Steerable Networks
• Transporter Networks: Rearranging the Visual World for Robotic Manipulation
• Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• LieTransformer: Equivariant self-attention for Lie Groups
• Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
• Scale-covariant and scale-invariant Gaussian derivative networks
• Attentive Group Equivariant Convolutional Networks
• Data-driven emergence of convolutional structure in neural networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Theoretical Aspects of Group Equivariant Neural Networks
• Meta-Learning Symmetries by Reparameterization
• Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes
• Parallel Transport Convolution: A New Tool for Convolutional Neural Networks on Manifolds
• Physics-guided Design and Learning of Neural Networks for Predicting Drag Force on Particle Suspensions in Moving Fluids
• Robustness of Rotation-Equivariant Networks to Adversarial Perturbations
• Set2Graph: Learning Graphs From Sets
• DeepSphere: towards an equivariant graph-based spherical CNN
• Symmetric Tensor Networks for Generative Modeling and Constrained Combinatorial Optimization
• Locality defeats the curse of dimensionality in convolutional teacher-student scenarios
• Temporal-Clustering Invariance in Irregular Healthcare Time Series
• A Data and Compute Efficient Design for Limited-Resources Deep Learning
• B-Spline CNNs on Lie Groups
• ManifoldNet: A Deep Network Framework for Manifold-valued Data
• A Simple Proof of the Universality of Invariant/Equivariant Graph Neural Networks
• Homogeneous vector bundles and $G$-equivariant convolutional neural networks
• On the Universality of Invariant Networks
• Equivariant Subgraph Aggregation Networks
• 3D-Rotation-Equivariant Quaternion Neural Networks
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Covariance in Physics and Convolutional Neural Networks
• Universal Equivariant Multilayer Perceptrons
• Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Quaternion Equivariant Capsule Networks for 3D Point Clouds
• Provably Strict Generalisation Benefit for Equivariant Models
• Spin-Weighted Spherical CNNs
• Group Invariant Dictionary Learning
• Symmetry-Aware Actor-Critic for 3D Molecular Design
• Category Theory in Machine Learning
• Reducing the dilution: An analysis of the information sensitiveness of capsule network with a practical improvement method
• A CNN for homogneous Riemannian manifolds with applications to Neuroimaging
• Quantum algorithms for group convolution, cross-correlation, and equivariant transformations
• DDNet: Cartesian-polar Dual-domain Network for the Joint Optic Disc and Cup Segmentation
• Universal Approximation Theorem for Equivariant Maps by Group CNNs
• Equivariant Maps for Hierarchical Structures
• MVC-Net: A Convolutional Neural Network Architecture for Manifold-Valued Images With Applications
• A contribution to Optimal Transport on incomparable spaces
• Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions
• Steerable Wavelet Scattering for 3D Atomic Systems with Application to Li-Si Energy Prediction
• Trajectory Prediction using Equivariant Continuous Convolution
• A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
• The general theory of permutation equivarant neural networks and higher order graph variational encoders
• Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Discrete Rotation Equivariance for Point Cloud Recognition
• SurReal: Complex-Valued Learning as Principled Transformations on a Scaling and Rotation Manifold
• Group Equivariant Stand-Alone Self-Attention For Vision
• Equivariant Entity-Relationship Networks
• Dilated Convolutional Neural Networks for Sequential Manifold-valued Data
• Robust Equivariant Imaging: a fully unsupervised framework for learning to image from noisy and partial measurements
• Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism
• Learning Compressed Transforms with Low Displacement Rank
• Group Equivariant Conditional Neural Processes
• Learning a Lie Algebra from Unlabeled Data Pairs
• Affine Self Convolution
• Rotation-Invariant Autoencoders for Signals on Spheres
• Prediction and Generalisation over Directed Actions by Grid Cells
• ManifoldNorm: Extending normalizations on Riemannian Manifolds
• Do CNNs Encode Data Augmentations?
• Flow-based Generative Models for Learning Manifold to Manifold Mappings
• Efficient Generalized Spherical CNNs
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• VolterraNet: A higher order convolutional network with group equivariance for homogeneous manifolds
• Autobahn: Automorphism-based Graph Neural Nets
• Equivariant Deep Dynamical Model for Motion Prediction
• Discriminative Learning of Similarity and Group Equivariant Representations
• 3D Solid Spherical Bispectrum CNNs for Biomedical Texture Analysis
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• A Bayesian Approach to Invariant Deep Neural Networks
• Implicit Bias of Linear Equivariant Networks
• Unsupervised Co-Learning on $\mathcal{G}$-Manifolds Across Irreducible Representations
• Knowledge as Invariance -- History and Perspectives of Knowledge-augmented Machine Learning
• Distribution-Based Invariant Deep Networks for Learning Meta-Features
• Equivariant Manifold Flows
• Computational Analysis of Deformable Manifolds: from Geometric Modelling to Deep Learning
• Deep Autoencoders: From Understanding to Generalization Guarantees
• Provably Strict Generalisation Benefit for Invariance in Kernel Methods
• Autoequivariant Network Search via Group Decomposition
• ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups
• Quantised Transforming Auto-Encoders: Achieving Equivariance to Arbitrary Transformations in Deep Networks