DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds

Abstract

Learning from 3D point-cloud data has rapidly gained momentum, motivated by the success of deep learning on images and the increased availability of 3D~data. In this paper, we aim to construct anisotropic convolution layers that work directly on the surface derived from a point cloud. This is challenging because of the lack of a global coordinate system for tangential directions on surfaces. We introduce DeltaConv, a convolution layer that combines geometric operators from vector calculus to enable the construction of anisotropic filters on point clouds. Because these operators are defined on scalar- and vector-fields, we separate the network into a scalar- and a vector-stream, which are connected by the operators. The vector stream enables the network to explicitly represent, evaluate, and process directional information. Our convolutions are robust and simple to implement and match or improve on state-of-the-art approaches on several benchmarks, while also speeding up training and inference.

arXiv comment8 pages, 5 figures, 7 tables; ACM Transactions on Graphics 41, 4, Article 105 (SIGGRAPH 2022)

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Geometric deep learning: going beyond Euclidean data
• Spectral Networks and Locally Connected Networks on Graphs
• ShapeNet: An Information-Rich 3D Model Repository
• MeshCNN: A Network with an Edge
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Geodesics in Heat
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Linked Dynamic Graph CNN: Learning on Point Cloud via Linking Hierarchical Features
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• Primal-Dual Mesh Convolutional Neural Networks
• 3DTI-Net: Learn Inner Transform Invariant 3D Geometry Features using Dynamic GCN
• DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Pooling
• SRINet: Learning Strictly Rotation-Invariant Representations for Point Cloud Classification and Segmentation
• DiffusionNet: Discretization Agnostic Learning on Surfaces
• MeshWalker: Deep Mesh Understanding by Random Walks
• HodgeNet: Learning Spectral Geometry on Triangle Meshes

Cited by in corpus (2)

• A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation
• PoissonNet: A Local-Global Approach for Learning on Surfaces