Spin-Weighted Spherical CNNs

Abstract

Learning equivariant representations is a promising way to reduce sample and model complexity and improve the generalization performance of deep neural networks. The spherical CNNs are successful examples, producing SO(3)-equivariant representations of spherical inputs. There are two main types of spherical CNNs. The first type lifts the inputs to functions on the rotation group SO(3) and applies convolutions on the group, which are computationally expensive since SO(3) has one extra dimension. The second type applies convolutions directly on the sphere, which are limited to zonal (isotropic) filters, and thus have limited expressivity. In this paper, we present a new type of spherical CNN that allows anisotropic filters in an efficient way, without ever leaving the spherical domain. The key idea is to consider spin-weighted spherical functions, which were introduced in physics in the study of gravitational waves. These are complex-valued functions on the sphere whose phases change upon rotation. We define a convolution between spin-weighted functions and build a CNN based on it. The spin-weighted functions can also be interpreted as spherical vector fields, allowing applications to tasks where the inputs or outputs are vector fields. Experiments show that our method outperforms previous methods on tasks like classification of spherical images, classification of 3D shapes and semantic segmentation of spherical panoramas.

arXiv commentAccepted to NeurIPS'20

References in corpus (20)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
• Rotation equivariant vector field networks
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• Exploiting Cyclic Symmetry in Convolutional Neural Networks
• Spherical CNNs
• Angular velocity of gravitational radiation from precessing binaries and the corotating frame
• General $E(2)$-Equivariant Steerable CNNs
• Covariant Compositional Networks For Learning Graphs
• 3D G-CNNs for Pulmonary Nodule Detection
• Spherical CNNs on Unstructured Grids
• Deep Scale-spaces: Equivariance Over Scale
• Fast and Exact Spin-s Spherical Harmonic Transforms
• How should spin-weighted spherical functions be defined?
• Theoretical Aspects of Group Equivariant Neural Networks
• Fast, exact (but unstable) spin spherical harmonic transforms

Cited by in corpus (5)

• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Equivariant Networks for Pixelized Spheres
• Efficient Generalized Spherical CNNs
• Group Equivariant Subsampling