Patch-based Progressive 3D Point Set Upsampling

Abstract

We present a detail-driven deep neural network for point set upsampling. A high-resolution point set is essential for point-based rendering and surface reconstruction. Inspired by the recent success of neural image super-resolution techniques, we progressively train a cascade of patch-based upsampling networks on different levels of detail end-to-end. We propose a series of architectural design contributions that lead to a substantial performance boost. The effect of each technical contribution is demonstrated in an ablation study. Qualitative and quantitative experiments show that our method significantly outperforms the state-of-the-art learning-based and optimazation-based approaches, both in terms of handling low-resolution inputs and revealing high-fidelity details.

arXiv commentaccepted to cvpr2019, code available at https://github.com/yifita/P3U

References in corpus (22)

• Conditional Generative Adversarial Nets
• Deep Residual Learning for Image Recognition
• Densely Connected Convolutional Networks
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• PCPNET: Learning Local Shape Properties from Raw Point Clouds
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
• Dynamic Graph CNN for Learning on Point Clouds
• Deep Networks with Stochastic Depth
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• SO-Net: Self-Organizing Network for Point Cloud Analysis
• 3D ShapeNets: A Deep Representation for Volumetric Shapes
• PU-Net: Point Cloud Upsampling Network
• Accurate Image Super-Resolution Using Very Deep Convolutional Networks
• Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
• Multiresolution Tree Networks for 3D Point Cloud Processing
• Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network
• PCN: Point Completion Network
• FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation
• Fully-Convolutional Point Networks for Large-Scale Point Clouds
• PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors
• Pointwise Convolutional Neural Networks

Cited by in corpus (3)

• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• 3D Point Cloud Processing and Learning for Autonomous Driving
• Parametric Surface Constrained Upsampler Network for Point Cloud