Conditional Single-view Shape Generation for Multi-view Stereo Reconstruction

Abstract

In this paper, we present a new perspective towards image-based shape generation. Most existing deep learning based shape reconstruction methods employ a single-view deterministic model which is sometimes insufficient to determine a single groundtruth shape because the back part is occluded. In this work, we first introduce a conditional generative network to model the uncertainty for single-view reconstruction. Then, we formulate the task of multi-view reconstruction as taking the intersection of the predicted shape spaces on each single image. We design new differentiable guidance including the front constraint, the diversity constraint, and the consistency loss to enable effective single-view conditional generation and multi-view synthesis. Experimental results and ablation studies show that our proposed approach outperforms state-of-the-art methods on 3D reconstruction test error and demonstrate its generalization ability on real world data.

14 pages, 13 figures, to appear in CVPR 2019. Code and data: https://github.com/weiyithu/OptimizeMVS

References in corpus (6)

• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Learning Representations and Generative Models for 3D Point Clouds
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

Cited by in corpus (3)

• DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
• Dense RepPoints: Representing Visual Objects with Dense Point Sets
• PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds