General $E(2)$-Equivariant Steerable CNNs

Abstract

The big empirical success of group equivariant networks has led in recent years to the sprouting of a great variety of equivariant network architectures. A particular focus has thereby been on rotation and reflection equivariant CNNs for planar images. Here we give a general description of $E(2)$-equivariant convolutions in the framework of Steerable CNNs. The theory of Steerable CNNs thereby yields constraints on the convolution kernels which depend on group representations describing the transformation laws of feature spaces. We show that these constraints for arbitrary group representations can be reduced to constraints under irreducible representations. A general solution of the kernel space constraint is given for arbitrary representations of the Euclidean group $E(2)$ and its subgroups. We implement a wide range of previously proposed and entirely new equivariant network architectures and extensively compare their performances. $E(2)$-steerable convolutions are further shown to yield remarkable gains on CIFAR-10, CIFAR-100 and STL-10 when used as a drop-in replacement for non-equivariant convolutions.

arXiv commentConference on Neural Information Processing Systems (NeurIPS), 2019

Cited by in corpus (59)

• Alias-Free Generative Adversarial Networks
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• PDE-based Group Equivariant Convolutional Neural Networks
• CoMIR: Contrastive Multimodal Image Representation for Registration
• MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Scale-Equivariant Steerable Networks
• ReDet: A Rotation-equivariant Detector for Aerial Object Detection
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• LieTransformer: Equivariant self-attention for Lie Groups
• Attentive Group Equivariant Convolutional Networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Learning Invariances in Neural Networks
• Theoretical Aspects of Group Equivariant Neural Networks
• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
• Meta-Learning Symmetries by Reparameterization
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Automatic Symmetry Discovery with Lie Algebra Convolutional Network
• FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection
• A Data and Compute Efficient Design for Limited-Resources Deep Learning
• Universal Equivariant Multilayer Perceptrons
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Spin-Weighted Spherical CNNs
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Group Equivariant Generative Adversarial Networks
• Equivariant Maps for Hierarchical Structures
• Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
• Probabilistic Numeric Convolutional Neural Networks
• Steerable Partial Differential Operators for Equivariant Neural Networks
• Partial local entropy and anisotropy in deep weight spaces
• Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent
• Trajectory Prediction using Equivariant Continuous Convolution
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Equivariant $Q$ Learning in Spatial Action Spaces
• Group Equivariant Stand-Alone Self-Attention For Vision
• Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
• Learnable Gabor modulated complex-valued networks for orientation robustness
• A simple equivariant machine learning method for dynamics based on scalars
• Variational Autoencoding of PDE Inverse Problems
• Group Equivariant Deep Reinforcement Learning
• Group Equivariant Conditional Neural Processes
• Equivariant neural networks for inverse problems
• Rotation Equivariant Deforestation Segmentation and Driver Classification
• Learning Augmentation Distributions using Transformed Risk Minimization
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• Implicit Equivariance in Convolutional Networks
• Deep Orthogonal Decompositions for Convective Nowcasting
• Rotating spiders and reflecting dogs: a class conditional approach to learning data augmentation distributions
• Group Equivariant Subsampling
• Training or Architecture? How to Incorporate Invariance in Neural Networks
• Nonlinearities in Steerable SO(2)-Equivariant CNNs
• E(2) Equivariant Self-Attention for Radio Astronomy
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• Implicit Bias of Linear Equivariant Networks
• FILTRA: Rethinking Steerable CNN by Filter Transform
• Autoequivariant Network Search via Group Decomposition
• Equivariance-bridged SO(2)-Invariant Representation Learning using Graph Convolutional Network