End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds
arXiv:2003.05855
Abstract
In this work, we propose an end-to-end framework to learn local multi-view descriptors for 3D point clouds. To adopt a similar multi-view representation, existing studies use hand-crafted viewpoints for rendering in a preprocessing stage, which is detached from the subsequent descriptor learning stage. In our framework, we integrate the multi-view rendering into neural networks by using a differentiable renderer, which allows the viewpoints to be optimizable parameters for capturing more informative local context of interest points. To obtain discriminative descriptors, we also design a soft-view pooling module to attentively fuse convolutional features across views. Extensive experiments on existing 3D registration benchmarks show that our method outperforms existing local descriptors both quantitatively and qualitatively.
CVPR 2020. Webpage: https://github.com/craigleili/3DLocalMultiViewDesc
References in corpus (12)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Instance Normalization: The Missing Ingredient for Fast Stylization
- In Defense of the Triplet Loss for Person Re-Identification
- Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
- 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration
- Differentiable Surface Splatting for Point-based Geometry Processing
- Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
- GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
- Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning
- 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
- Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer
- Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition