Spherical CNNs on Unstructured Grids

Abstract

We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals. To this end, we replace conventional convolution kernels with linear combinations of differential operators that are weighted by learnable parameters. Differential operators can be efficiently estimated on unstructured grids using one-ring neighbors, and learnable parameters can be optimized through standard back-propagation. As a result, we obtain extremely efficient neural networks that match or outperform state-of-the-art network architectures in terms of performance but with a significantly lower number of network parameters. We evaluate our algorithm in an extensive series of experiments on a variety of computer vision and climate science tasks, including shape classification, climate pattern segmentation, and omnidirectional image semantic segmentation. Overall, we present (1) a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals, and (2) we show that our unique kernel parameterization allows our model to achieve the same or higher accuracy with significantly fewer network parameters.

Accepted as a conference paper at ICLR 2019. Codes available at https://github.com/maxjiang93/ugscnn

Cited by in corpus (27)

• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• MeshCNN: A Network with an Edge
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• A Survey on Graph-Based Deep Learning for Computational Histopathology
• Cortical surface registration using unsupervised learning
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression
• DeepSphere: a graph-based spherical CNN
• DeepSphere: towards an equivariant graph-based spherical CNN
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Spin-Weighted Spherical CNNs
• Quaternion Equivariant Capsule Networks for 3D Point Clouds
• Wavelet-Based Segmentation on the Sphere
• HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features
• Orientation-aware Semantic Segmentation on Icosahedron Spheres
• Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs
• Graph Convolution with Low-rank Learnable Local Filters
• Shape As Points: A Differentiable Poisson Solver
• TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
• Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds
• Towards Unsupervised Segmentation of Extreme Weather Events
• Spherical Transformer: Adapting Spherical Signal to CNNs
• Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh
• Equivariant Networks for Pixelized Spheres
• Variable-Viewpoint Representations for 3D Object Recognition
• Learning Equivariant Representations
• Equivariant Point Network for 3D Point Cloud Analysis