Theoretical Aspects of Group Equivariant Neural Networks
arXiv:2004.05154
Abstract
Group equivariant neural networks have been explored in the past few years and are interesting from theoretical and practical standpoints. They leverage concepts from group representation theory, non-commutative harmonic analysis and differential geometry that do not often appear in machine learning. In practice, they have been shown to reduce sample and model complexity, notably in challenging tasks where input transformations such as arbitrary rotations are present. We begin this work with an exposition of group representation theory and the machinery necessary to define and evaluate integrals and convolutions on groups. Then, we show applications to recent SO(3) and SE(3) equivariant networks, namely the Spherical CNNs, Clebsch-Gordan Networks, and 3D Steerable CNNs. We proceed to discuss two recent theoretical results. The first, by Kondor and Trivedi (ICML'18), shows that a neural network is group equivariant if and only if it has a convolutional structure. The second, by Cohen et al. (NeurIPS'19), generalizes the first to a larger class of networks, with feature maps as fields on homogeneous spaces.
Corrected 3D steerable CNNs kernel characterization and other minor fixes
Cited by in corpus (6)
- Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
- SO(2)-Equivariant Downwash Models for Close Proximity Flight
- Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism
- Rotation-Invariant Autoencoders for Signals on Spheres
- Solving the QCD effective kinetic theory with neural networks
- Learning Equivariant Representations