Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Abstract

Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity. We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes. To this end, we propose an effective divide-and-conquer strategy that divides pixels based on their plane orientation estimation; then, the succeeding instance segmentation module conquers the task of planes clustering more easily in each plane orientation group. Besides, parameters of V-planes depend on camera yaw rotation, but translation-invariant CNNs are less aware of the yaw change. We thus propose a yaw-invariant V-planar reparameterization for CNNs to learn. We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H\&V-planes (referred to as PanoH&V dataset) and adopt state-of-the-art planar reconstruction methods to predict H\&V-planes as our baselines. Our method outperforms the baselines by a large margin on the proposed dataset.

arXiv commentCode at https://github.com/sunset1995/PanoPlane360. Video at https://www.youtube.com/watch?v=2uvP0V1oGRo

References in corpus (7)

• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Semantic Instance Segmentation with a Discriminative Loss Function
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Semantic Instance Segmentation via Deep Metric Learning
• Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling