OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas
arXiv:2104.09403
Abstract
Given a single RGB panorama, the goal of 3D layout reconstruction is to estimate the room layout by predicting the corners, floor boundary, and ceiling boundary. A common approach has been to use standard convolutional networks to predict the corners and boundaries, followed by post-processing to generate the 3D layout. However, the space-varying distortions in panoramic images are not compatible with the translational equivariance property of standard convolutions, thus degrading performance. Instead, we propose to use spherical convolutions. The resulting network, which we call OmniLayout performs convolutions directly on the sphere surface, sampling according to inverse equirectangular projection and hence invariant to equirectangular distortions. Using a new evaluation metric, we show that our network reduces the error in the heavily distorted regions (near the poles) by approx 25 % when compared to standard convolutional networks. Experimental results show that OmniLayout outperforms the state-of-the-art by approx 4% on two different benchmark datasets (PanoContext and Stanford 2D-3D). Code is available at https://github.com/rshivansh/OmniLayout.
arXiv commentAccepted at CVPR, OmniCV Workshop. 10 Pages, 9 Figures, 6 Tables
References in corpus (12)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Deep Residual Learning for Image Recognition
- Joint 2D-3D-Semantic Data for Indoor Scene Understanding
- Predicting Complete 3D Models of Indoor Scenes
- Noisy Student Training using Body Language Dataset Improves Facial Expression Recognition
- Geometric deep learning on graphs and manifolds using mixture model CNNs
- LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image
- HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation
- HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features
- DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
- Pano2CAD: Room Layout From A Single Panorama Image
- Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation