Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)

Abstract

Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of scalar fields ("feature channels"), whereas the steerable G-CNN can also use vector or tensor fields ("capsules") to represent data. In algebraic terms, the feature spaces in regular G-CNNs transform according to a regular representation of the group G, whereas the feature spaces in Steerable G-CNNs transform according to the more general induced representations of G. In order to make the network equivariant, each layer in a G-CNN is required to intertwine between the induced representations associated with its input and output space. In this paper we present a general mathematical framework for G-CNNs on homogeneous spaces like Euclidean space or the sphere. We show, using elementary methods, that the layers of an equivariant network are convolutional if and only if the input and output feature spaces transform according to an induced representation. This result, which follows from G.W. Mackey's abstract theory on induced representations, establishes G-CNNs as a universal class of equivariant network architectures, and generalizes the important recent work of Kondor & Trivedi on the intertwiners between regular representations.

References in corpus (5)

• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Spherical CNNs
• Covariant Compositional Networks For Learning Graphs
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Learning the Irreducible Representations of Commutative Lie Groups

Cited by in corpus (15)

• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• 3D G-CNNs for Pulmonary Nodule Detection
• Explorations in Homeomorphic Variational Auto-Encoding
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• not-MIWAE: Deep Generative Modelling with Missing not at Random Data
• Covariance in Physics and Convolutional Neural Networks
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• AVT: Unsupervised Learning of Transformation Equivariant Representations by Autoencoding Variational Transformations
• Discrete Rotation Equivariance for Point Cloud Recognition
• Equivariant Entity-Relationship Networks
• Group Equivariant Subsampling
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image