DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

Abstract

Reconstructing 3D shapes from single-view images has been a long-standing research problem. In this paper, we present DISN, a Deep Implicit Surface Network which can generate a high-quality detail-rich 3D mesh from an 2D image by predicting the underlying signed distance fields. In addition to utilizing global image features, DISN predicts the projected location for each 3D point on the 2D image, and extracts local features from the image feature maps. Combining global and local features significantly improves the accuracy of the signed distance field prediction, especially for the detail-rich areas. To the best of our knowledge, DISN is the first method that constantly captures details such as holes and thin structures present in 3D shapes from single-view images. DISN achieves the state-of-the-art single-view reconstruction performance on a variety of shape categories reconstructed from both synthetic and real images. Code is available at https://github.com/xharlie/DISN The supplementary can be found at https://xharlie.github.io/images/neurips_2019_supp.pdf

This project was in part supported by the gift funding to the University of Southern California from Adobe Research

References in corpus (13)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• MarrNet: 3D Shape Reconstruction via 2.5D Sketches
• Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
• Learning Implicit Fields for Generative Shape Modeling
• On the Continuity of Rotation Representations in Neural Networks
• Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
• SurfNet: Generating 3D shape surfaces using deep residual networks
• Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks
• Learning Shape Priors for Single-View 3D Completion and Reconstruction
• Im2Struct: Recovering 3D Shape Structure from a Single RGB Image
• 3DN: 3D Deformation Network
• A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images

Cited by in corpus (11)

• Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images
• Joint Implicit Image Function for Guided Depth Super-Resolution
• GIFS: Neural Implicit Function for General Shape Representation
• NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Dropping
• Learning Mesh Representations via Binary Space Partitioning Tree Networks
• Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image
• SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorization
• MARF: The Medial Atom Ray Field Object Representation
• Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
• Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
• Single-view 3D Body and Cloth Reconstruction under Complex Poses