A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups

Abstract

Symmetries and equivariance are fundamental to the generalization of neural networks on domains such as images, graphs, and point clouds. Existing work has primarily focused on a small number of groups, such as the translation, rotation, and permutation groups. In this work we provide a completely general algorithm for solving for the equivariant layers of matrix groups. In addition to recovering solutions from other works as special cases, we construct multilayer perceptrons equivariant to multiple groups that have never been tackled before, including $\mathrm{O}(1,3)$, $\mathrm{O}(5)$, $\mathrm{Sp}(n)$, and the Rubik's cube group. Our approach outperforms non-equivariant baselines, with applications to particle physics and dynamical systems. We release our software library to enable researchers to construct equivariant layers for arbitrary matrix groups.

Library: https://github.com/mfinzi/equivariant-MLP, Documentation: https://emlp.readthedocs.io/en/latest/, Examples: https://colab.research.google.com/github/mfinzi/equivariant-MLP/blob/master/docs/notebooks/colabs/all.ipynb

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Searching for Activation Functions
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Rotation equivariant vector field networks
• Equivariance Through Parameter-Sharing
• General $E(2)$-Equivariant Steerable CNNs
• Lorentz Group Equivariant Neural Network for Particle Physics
• Fast and Simple PCA via Convex Optimization
• On Learning Sets of Symmetric Elements
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• Polar Transformer Networks
• On the Universality of Rotation Equivariant Point Cloud Networks
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Set2Graph: Learning Graphs From Sets
• B-Spline CNNs on Lie Groups
• Universal Equivariant Multilayer Perceptrons
• Equivariant Maps for Hierarchical Structures
• Dense Random Finitely Generated Subgroups of Lie Groups

Cited by in corpus (14)

• A method to challenge symmetries in data with self-supervised learning
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Generative modeling of living cells with SO(3)-equivariant implicit neural representations
• On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis
• Topographic VAEs learn Equivariant Capsules
• Complete Neural Networks for Complete Euclidean Graphs
• A simple equivariant machine learning method for dynamics based on scalars
• Robustness through Data Augmentation Loss Consistency
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Autoequivariant Network Search via Group Decomposition
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• Geometric Algebra Attention Networks for Small Point Clouds
• Symmetry-driven graph neural networks