Learning Shape Priors for Single-View 3D Completion and Reconstruction

Abstract

The problem of single-view 3D shape completion or reconstruction is challenging, because among the many possible shapes that explain an observation, most are implausible and do not correspond to natural objects. Recent research in the field has tackled this problem by exploiting the expressiveness of deep convolutional networks. In fact, there is another level of ambiguity that is often overlooked: among plausible shapes, there are still multiple shapes that fit the 2D image equally well; i.e., the ground truth shape is non-deterministic given a single-view input. Existing fully supervised approaches fail to address this issue, and often produce blurry mean shapes with smooth surfaces but no fine details. In this paper, we propose ShapeHD, pushing the limit of single-view shape completion and reconstruction by integrating deep generative models with adversarially learned shape priors. The learned priors serve as a regularizer, penalizing the model only if its output is unrealistic, not if it deviates from the ground truth. Our design thus overcomes both levels of ambiguity aforementioned. Experiments demonstrate that ShapeHD outperforms state of the art by a large margin in both shape completion and shape reconstruction on multiple real datasets.

arXiv commentECCV 2018. The first two authors contributed equally to this work. Project page: http://shapehd.csail.mit.edu/

References in corpus (10)

• Deep Residual Learning for Image Recognition
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Improved Training of Wasserstein GANs
• Object Detectors Emerge in Deep Scene CNNs
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• Single-Image Depth Perception in the Wild
• Unsupervised Learning of 3D Structure from Images
• Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
• Hierarchical Surface Prediction for 3D Object Reconstruction
• OctNetFusion: Learning Depth Fusion from Data

Cited by in corpus (5)

• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
• Shallow2Deep: Indoor Scene Modeling by Single Image Understanding
• Learning View Priors for Single-view 3D Reconstruction
• Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction
• Single Image 3D Object Estimation with Primitive Graph Networks