Universal Equivariant Multilayer Perceptrons

Abstract

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant MLPs with a single hidden layer. In particular, it is shown that having a hidden layer on which the group acts regularly is sufficient for universal equivariance (invariance). A corollary is unconditional universality of equivariant MLPs for Abelian groups, such as CNNs with a single hidden layer. A second corollary is the universality of equivariant MLPs with a high-order hidden layer, where we give both group-agnostic bounds and means for calculating group-specific bounds on the order of hidden layer that guarantees universal equivariance (invariance).

References in corpus (3)

• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Covariant Compositional Networks For Learning Graphs
• Deep Learning with Sets and Point Clouds

Cited by in corpus (8)

• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
• On the Universality of Rotation Equivariant Point Cloud Networks
• Expressive Power of Invariant and Equivariant Graph Neural Networks
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Universal Approximation Theorem for Equivariant Maps by Group CNNs
• Group Equivariant Stand-Alone Self-Attention For Vision
• Meta-Learning for One-Class Classification with Few Examples using Order-Equivariant Network