3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data

Abstract

We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equivariant convolutions utilize kernels which are parameterized as a linear combination of a complete steerable kernel basis, which is derived analytically in this paper. We prove that equivariant convolutions are the most general equivariant linear maps between fields over R^3. Our experimental results confirm the effectiveness of 3D Steerable CNNs for the problem of amino acid propensity prediction and protein structure classification, both of which have inherent SE(3) symmetry.

References in corpus (11)

• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Why do deep convolutional networks generalize so poorly to small image transformations?
• Spherical CNNs
• Equivariance Through Parameter-Sharing
• Covariant Compositional Networks For Learning Graphs
• 3D G-CNNs for Pulmonary Nodule Detection
• N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
• Learning the Irreducible Representations of Commutative Lie Groups
• Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network
• Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)

Cited by in corpus (67)

• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Cormorant: Covariant Molecular Neural Networks
• Machine Learning on Neutron and X-Ray Scattering
• A Rotation-Invariant Framework for Deep Point Cloud Analysis
• Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry
• SchNetPack 2.0: A neural network toolbox for atomistic machine learning
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• On Learning Sets of Symmetric Elements
• Rotation Invariant Graph Neural Networks using Spin Convolutions
• Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties
• MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Fanaroff-Riley classification of radio galaxies using group-equivariant convolutional neural networks
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• LieTransformer: Equivariant self-attention for Lie Groups
• Attentive Group Equivariant Convolutional Networks
• Standardised convolutional filtering for radiomics
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Data-Driven Approach to Encoding and Decoding 3-D Crystal Structures
• Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory
• Set2Graph: Learning Graphs From Sets
• Homogeneous vector bundles and $G$-equivariant convolutional neural networks
• B-Spline CNNs on Lie Groups
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI
• Frame Averaging for Invariant and Equivariant Network Design
• Quaternion Equivariant Capsule Networks for 3D Point Clouds
• Spin-Weighted Spherical CNNs
• Symmetry-Aware Actor-Critic for 3D Molecular Design
• Rotational 3D Texture Classification Using Group Equivariant CNNs
• Orientation-aware Semantic Segmentation on Icosahedron Spheres
• Embed Me If You Can: A Geometric Perceptron
• Disentangling by Subspace Diffusion
• Steerable Partial Differential Operators for Equivariant Neural Networks
• Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds
• The general theory of permutation equivarant neural networks and higher order graph variational encoders
• Equivariant Entity-Relationship Networks
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Group Equivariant Stand-Alone Self-Attention For Vision
• Isometric Transformation Invariant and Equivariant Graph Convolutional Networks
• Volumetric Convolution: Automatic Representation Learning in Unit Ball
• Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods
• Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder
• Understanding the Generalization Benefit of Model Invariance from a Data Perspective
• Quaternion Product Units for Deep Learning on 3D Rotation Groups
• Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space
• Scale Equivariant Neural Networks with Morphological Scale-Spaces
• Equivariant neural networks for inverse problems
• USCO-Solver: Solving Undetermined Stochastic Combinatorial Optimization Problems
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Equivariant Deep Dynamical Model for Motion Prediction
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• Implicit Equivariance in Convolutional Networks
• Rotational and Reflectional Equivariant Convolutional Neural Network for data-limited applications: Multiphase Flow demonstration
• Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi^4 field theory
• Learning Similarity Metrics for Volumetric Simulations with Multiscale CNNs
• Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image
• FILTRA: Rethinking Steerable CNN by Filter Transform
• Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups
• Computing equivariant matrices on homogeneous spaces for Geometric Deep Learning and Automorphic Lie Algebras