PointCNN: Convolution On -Transformed Points
arXiv:1801.07791
Abstract
We present a simple and general framework for feature learning from point clouds. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point clouds are irregular and unordered, thus directly convolving kernels against features associated with the points, will result in desertion of shape information and variance to point ordering. To address these problems, we propose to learn an -transformation from the input points, to simultaneously promote two causes. The first is the weighting of the input features associated with the points, and the second is the permutation of the points into a latent and potentially canonical order. Element-wise product and sum operations of the typical convolution operator are subsequently applied on the -transformed features. The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.
To be published in NIPS 2018, code available at https://github.com/yangyanli/PointCNN
References in corpus (14)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- A Neural Representation of Sketch Drawings
- Submanifold Sparse Convolutional Networks
- Dynamic Graph CNN for Learning on Point Clouds
- SO-Net: Self-Organizing Network for Point Cloud Analysis
- 3D ShapeNets: A Deep Representation for Volumetric Shapes
- SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters
- SPLATNet: Sparse Lattice Networks for Point Cloud Processing
- Local Spectral Graph Convolution for Point Set Feature Learning
- Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
- 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
- Tangent Convolutions for Dense Prediction in 3D
- H-CNN: Spatial Hashing Based CNN for 3D Shape Analysis
- Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)
Cited by in corpus (29)
- Linked Dynamic Graph CNN: Learning on Point Cloud via Linking Hierarchical Features
- Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scans
- PointConv: Deep Convolutional Networks on 3D Point Clouds
- PU-Net: Point Cloud Upsampling Network
- 3D shape sensing and deep learning-based segmentation of strawberries
- Iterative Transformer Network for 3D Point Cloud
- Patch-based Progressive 3D Point Set Upsampling
- Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network
- PCN: Point Completion Network
- PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
- PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
- Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution
- FPConv: Learning Local Flattening for Point Convolution
- Learning Material-Aware Local Descriptors for 3D Shapes
- Pointwise Convolutional Neural Networks
- Real-time Progressive 3D Semantic Segmentation for Indoor Scene
- Point2Node: Correlation Learning of Dynamic-Node for Point Cloud Feature Modeling
- Adaptive Hierarchical Down-Sampling for Point Cloud Classification
- Continuous Geodesic Convolutions for Learning on 3D Shapes
- View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors
- Discrete Rotation Equivariance for Point Cloud Recognition
- PVRNet: Point-View Relation Neural Network for 3D Shape Recognition
- CAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision
- Point Cloud Audio Processing
- Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Network
- Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction
- LO-Net: Deep Real-time Lidar Odometry
- 3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds
- Dense Graph Convolutional Neural Networks on 3D Meshes for 3D Object Segmentation and Classification