PU-Net: Point Cloud Upsampling Network

Abstract

Learning and analyzing 3D point clouds with deep networks is challenging due to the sparseness and irregularity of the data. In this paper, we present a data-driven point cloud upsampling technique. The key idea is to learn multi-level features per point and expand the point set via a multi-branch convolution unit implicitly in feature space. The expanded feature is then split to a multitude of features, which are then reconstructed to an upsampled point set. Our network is applied at a patch-level, with a joint loss function that encourages the upsampled points to remain on the underlying surface with a uniform distribution. We conduct various experiments using synthesis and scan data to evaluate our method and demonstrate its superiority over some baseline methods and an optimization-based method. Results show that our upsampled points have better uniformity and are located closer to the underlying surfaces.

accepted by CVPR2018

References in corpus (3)

• Fully Convolutional Networks for Semantic Segmentation
• PCPNET: Learning Local Shape Properties from Raw Point Clouds
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points

Cited by in corpus (17)

• A Rotation-Invariant Framework for Deep Point Cloud Analysis
• DNF-Net: a Deep Normal Filtering Network for Mesh Denoising
• PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
• VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation
• Patch-based Progressive 3D Point Set Upsampling
• 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image
• PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN
• JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds
• Frequency-Selective Mesh-to-Mesh Resampling for Color Upsampling of Point Clouds
• Pointwise Convolutional Neural Networks
• Parametric Surface Constrained Upsampler Network for Point Cloud
• BIMS-PU: Bi-Directional and Multi-Scale Point Cloud Upsampling
• Going Deeper with Lean Point Networks
• Fast Generation of High Fidelity RGB-D Images by Deep-Learning with Adaptive Convolution
• 3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds