Gauge Equivariant Convolutional Networks and the Icosahedral CNN

Abstract

The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be extended beyond global symmetries to local gauge transformations. This enables the development of a very general class of convolutional neural networks on manifolds that depend only on the intrinsic geometry, and which includes many popular methods from equivariant and geometric deep learning. We implement gauge equivariant CNNs for signals defined on the surface of the icosahedron, which provides a reasonable approximation of the sphere. By choosing to work with this very regular manifold, we are able to implement the gauge equivariant convolution using a single conv2d call, making it a highly scalable and practical alternative to Spherical CNNs. Using this method, we demonstrate substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns.

arXiv commentProceedings of the International Conference on Machine Learning (ICML), 2019

References in corpus (14)

• Geometric deep learning: going beyond Euclidean data
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Rotation equivariant vector field networks
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• Equivariance Through Parameter-Sharing
• Invariant and Equivariant Graph Networks
• 3D G-CNNs for Pulmonary Nodule Detection
• Spherical CNNs on Unstructured Grids
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
• Parallel Transport Convolution: A New Tool for Convolutional Neural Networks on Manifolds

Cited by in corpus (51)

• Machine learning and the physical sciences
• A Survey of Sound Source Localization with Deep Learning Methods
• Flow-based generative models for Markov chain Monte Carlo in lattice field theory
• Stand-Alone Self-Attention in Vision Models
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Cormorant: Covariant Molecular Neural Networks
• Sampling using $SU(N)$ gauge equivariant flows
• Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics
• CNNs on Surfaces using Rotation-Equivariant Features
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• On Learning Sets of Symmetric Elements
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Scale-Equivariant Steerable Networks
• Permutationless Many-Jet Event Reconstruction with Symmetry Preserving Attention Networks
• DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds
• Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs
• On the "steerability" of generative adversarial networks
• Attentive Group Equivariant Convolutional Networks
• Data-driven emergence of convolutional structure in neural networks
• Nonlinear 3D Cosmic Web Simulation with Heavy-Tailed Generative Adversarial Networks
• Scrambling and decoding the charged quantum information
• Meta-Learning Symmetries by Reparameterization
• Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
• InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation
• Introduction to Normalizing Flows for Lattice Field Theory
• Rotation-Equivariant Deep Learning for Diffusion MRI
• Universal Equivariant Multilayer Perceptrons
• Machine learning and invariant theory
• Covariance in Physics and Convolutional Neural Networks
• Category Theory in Machine Learning
• Orientation-aware Semantic Segmentation on Icosahedron Spheres
• Equivariance and generalization in neural networks
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Topos and Stacks of Deep Neural Networks
• Equivariant Entity-Relationship Networks
• Group Equivariant Stand-Alone Self-Attention For Vision
• ManifoldNorm: Extending normalizations on Riemannian Manifolds
• Rotation-Invariant Autoencoders for Signals on Spheres
• Affine Self Convolution
• The Physics of Machine Learning: An Intuitive Introduction for the Physical Scientist
• RISA-Net: Rotation-Invariant Structure-Aware Network for Fine-Grained 3D Shape Retrieval
• Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh
• Equivariant Networks for Pixelized Spheres
• Preserving gauge invariance in neural networks
• Efficient Generalized Spherical CNNs
• ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups
• Computing equivariant matrices on homogeneous spaces for Geometric Deep Learning and Automorphic Lie Algebras
• Surrogate Modelling for Injection Molding Processes using Machine Learning
• Interpretable Image Clustering via Diffeomorphism-Aware K-Means
• Equivariant Point Network for 3D Point Cloud Analysis
• Computational Analysis of Deformable Manifolds: from Geometric Modelling to Deep Learning