HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation

Abstract

We present a new approach to the problem of estimating the 3D room layout from a single panoramic image. We represent room layout as three 1D vectors that encode, at each image column, the boundary positions of floor-wall and ceiling-wall, and the existence of wall-wall boundary. The proposed network, HorizonNet, trained for predicting 1D layout, outperforms previous state-of-the-art approaches. The designed post-processing procedure for recovering 3D room layouts from 1D predictions can automatically infer the room shape with low computation cost - it takes less than 20ms for a panorama image while prior works might need dozens of seconds. We also propose Pano Stretch Data Augmentation, which can diversify panorama data and be applied to other panorama-related learning tasks. Due to the limited data available for non-cuboid layout, we relabel 65 general layout from the current dataset for finetuning. Our approach shows good performance on general layouts by qualitative results and cross-validation.

arXiv commentCVPR 2019

References in corpus (4)

• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image
• PanoRoom: From the Sphere to the 3D Layout
• DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama

Cited by in corpus (5)

• Flat2Layout: Flat Representation for Estimating Layout of General Room Types
• SceneGen: Generative Contextual Scene Augmentation using Scene Graph Priors
• COCO-GAN: Generation by Parts via Conditional Coordinating
• Monte Carlo Scene Search for 3D Scene Understanding
• OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas