Tangent Convolutions for Dense Prediction in 3D

Abstract

We present an approach to semantic scene analysis using deep convolutional networks. Our approach is based on tangent convolutions - a new construction for convolutional networks on 3D data. In contrast to volumetric approaches, our method operates directly on surface geometry. Crucially, the construction is applicable to unstructured point clouds and other noisy real-world data. We show that tangent convolutions can be evaluated efficiently on large-scale point clouds with millions of points. Using tangent convolutions, we design a deep fully-convolutional network for semantic segmentation of 3D point clouds, and apply it to challenging real-world datasets of indoor and outdoor 3D environments. Experimental results show that the presented approach outperforms other recent deep network constructions in detailed analysis of large 3D scenes.

References in corpus (12)

• TensorFlow: A system for large-scale machine learning
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Open3D: A Modern Library for 3D Data Processing
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• FPNN: Field Probing Neural Networks for 3D Data
• Learning shape correspondence with anisotropic convolutional neural networks
• VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
• 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks
• Hierarchical Surface Prediction for 3D Object Reconstruction
• OctNetFusion: Learning Depth Fusion from Data

Cited by in corpus (16)

• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation
• PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation
• Deep Learning for 3D Point Cloud Understanding: A Survey
• PFCNN: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames
• Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• Point2Node: Correlation Learning of Dynamic-Node for Point Cloud Feature Modeling
• FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
• A Fast Point Cloud Ground Segmentation Approach Based on Coarse-To-Fine Markov Random Field
• Visualizing Point Cloud Classifiers by Curvature Smoothing
• Multi Voxel-Point Neurons Convolution (MVPConv) for Fast and Accurate 3D Deep Learning
• The Devils in the Point Clouds: Studying the Robustness of Point Cloud Convolutions
• Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving
• Learning Propagation for Arbitrarily-structured Data