Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network

Abstract

In this work we present a method to train a plane-aware convolutional neural network for dense depth and surface normal estimation as well as plane boundaries from a single indoor $360^{\circ }$ image. Using our proposed loss function, our network outperforms existing methods for single-view, indoor, omnidirectional depth estimation and provides an initial benchmark for surface normal prediction from $360^{\circ }$ images. Our improvements are due to the use of a novel plane-aware loss that leverages principal curvature as an indicator of planar boundaries. We also show that including geodesic coordinate maps as network priors provides a significant boost in surface normal prediction accuracy. Finally, we demonstrate how we can combine our network's outputs to generate high quality 3D "pop-up" models of indoor scenes.

arXiv comment2019 International Conference on 3D Vision (3DV). IEEE, 2019

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Learning Spherical Convolution for Fast Features from 360° Imagery
• Convolutional Networks for Spherical Signals
• Convolutions on Spherical Images
• Joint Prediction of Depths, Normals and Surface Curvature from RGB Images using CNNs
• A Fast Method For Computing Principal Curvatures From Range Images