MVPNet: Multi-View Point Regression Networks for 3D Object Reconstruction from A Single Image

Abstract

In this paper, we address the problem of reconstructing an object's surface from a single image using generative networks. First, we represent a 3D surface with an aggregation of dense point clouds from multiple views. Each point cloud is embedded in a regular 2D grid aligned on an image plane of a viewpoint, making the point cloud convolution-favored and ordered so as to fit into deep network architectures. The point clouds can be easily triangulated by exploiting connectivities of the 2D grids to form mesh-based surfaces. Second, we propose an encoder-decoder network that generates such kind of multiple view-dependent point clouds from a single image by regressing their 3D coordinates and visibilities. We also introduce a novel geometric loss that is able to interpret discrepancy over 3D surfaces as opposed to 2D projective planes, resorting to the surface discretization on the constructed meshes. We demonstrate that the multi-view point regression network outperforms state-of-the-art methods with a significant improvement on challenging datasets.

arXiv comment8 pages; accepted by AAAI 2019

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
• Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction