Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
arXiv:1712.06760
Abstract
Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, PointNet has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with a more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy to a convolution kernel for images, we define a point-set kernel as a set of learnable 3D points that jointly respond to a set of neighboring data points according to their geometric affinities measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local high-dimensional feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network can efficiently capture local information and robustly achieve better performances on major datasets. Our code is available at http://www.merl.com/research/license#KCNet
arXiv commentAccepted in CVPR'18. *indicates equal contribution
References in corpus (14)
- Semi-Supervised Classification with Graph Convolutional Networks
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Geometric deep learning: going beyond Euclidean data
- Spectral Networks and Locally Connected Networks on Graphs
- Deep Convolutional Networks on Graph-Structured Data
- Learning Convolutional Neural Networks for Graphs
- ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
- Diffusion-Convolutional Neural Networks
- Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
- FPNN: Field Probing Neural Networks for 3D Data
- 3D ShapeNets: A Deep Representation for Volumetric Shapes
- Graph Based Convolutional Neural Network
- OctNet: Learning Deep 3D Representations at High Resolutions
- FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation
Cited by in corpus (9)
- Dynamic Graph CNN for Learning on Point Clouds
- Patch-based Progressive 3D Point Set Upsampling
- PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
- PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
- PointNetLK: Robust & Efficient Point Cloud Registration using PointNet
- Inferring Point Clouds from Single Monocular Images by Depth Intermediation
- PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
- MeshNet: Mesh Neural Network for 3D Shape Representation
- A Deeper Look at 3D Shape Classifiers