3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Abstract

Inspired by the recent success of methods that employ shape priors to achieve robust 3D reconstructions, we propose a novel recurrent neural network architecture that we call the 3D Recurrent Reconstruction Neural Network (3D-R2N2). The network learns a mapping from images of objects to their underlying 3D shapes from a large collection of synthetic data. Our network takes in one or more images of an object instance from arbitrary viewpoints and outputs a reconstruction of the object in the form of a 3D occupancy grid. Unlike most of the previous works, our network does not require any image annotations or object class labels for training or testing. Our extensive experimental analysis shows that our reconstruction framework i) outperforms the state-of-the-art methods for single view reconstruction, and ii) enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).

arXiv commentAppendix can be found at http://cvgl.stanford.edu/papers/choy_16_appendix.pdf

References in corpus (2)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• A Large Dataset of Object Scans

Cited by in corpus (10)

• Unsupervised Learning of 3D Structure from Images
• A Point Set Generation Network for 3D Object Reconstruction from a Single Image
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
• IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
• Grid-GCN for Fast and Scalable Point Cloud Learning
• 3D Reconstruction of Simple Objects from A Single View Silhouette Image
• Semantic Photometric Bundle Adjustment on Natural Sequences
• Learning a Hierarchical Latent-Variable Model of 3D Shapes
• 3D Face Reconstruction by Learning from Synthetic Data