3D Surface Reconstruction by Pointillism
arXiv:1809.02002
Abstract
The objective of this work is to infer the 3D shape of an object from a single image. We use sculptures as our training and test bed, as these have great variety in shape and appearance. To achieve this we build on the success of multiple view geometry (MVG) which is able to accurately provide correspondences between images of 3D objects under varying viewpoint and illumination conditions, and make the following contributions: first, we introduce a new loss function that can harness image-to-image correspondences to provide a supervisory signal to train a deep network to infer a depth map. The network is trained end-to-end by differentiating through the camera. Second, we develop a processing pipeline to automatically generate a large scale multi-view set of correspondences for training the network. Finally, we demonstrate that we can indeed obtain a depth map of a novel object from a single image for a variety of sculptures with varying shape/texture, and that the network generalises at test time to new domains (e.g. synthetic images).
ECCV workshop on Geometry meets Deep Learning
References in corpus (8)
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
- Open3D: A Modern Library for 3D Data Processing
- SfM-Net: Learning of Structure and Motion from Video
- MegaDepth: Learning Single-View Depth Prediction from Internet Photos
- Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
- SurfNet: Generating 3D shape surfaces using deep residual networks
- 3D Shape Induction from 2D Views of Multiple Objects