Spherical View Synthesis for Self-Supervised 360 Depth Estimation

Abstract

Learning based approaches for depth perception are limited by the availability of clean training data. This has led to the utilization of view synthesis as an indirect objective for learning depth estimation using efficient data acquisition procedures. Nonetheless, most research focuses on pinhole based monocular vision, with scarce works presenting results for omnidirectional input. In this work, we explore spherical view synthesis for learning monocular 360 depth in a self-supervised manner and demonstrate its feasibility. Under a purely geometrically derived formulation we present results for horizontal and vertical baselines, as well as for the trinocular case. Further, we show how to better exploit the expressiveness of traditional CNNs when applied to the equirectangular domain in an efficient manner. Finally, given the availability of ground truth depth data, our work is uniquely positioned to compare view synthesis against direct supervision in a consistent and fair manner. The results indicate that alternative research directions might be better suited to enable higher quality depth perception. Our data, models and code are publicly available at https://vcl3d.github.io/SphericalViewSynthesis/.

arXiv comment3DV19, code and data at https://vcl3d.github.io/SphericalViewSynthesis/

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Adaptive Gradient Methods with Dynamic Bound of Learning Rate
• Learning Spherical Convolution for Fast Features from 360° Imagery
• Deep Residual Networks with Exponential Linear Unit
• Kernel Transformer Networks for Compact Spherical Convolution
• Single View Stereo Matching
• Self-Supervised Learning of Depth and Camera Motion from 360° Videos
• Learning structure-from-motion from motion

Cited by in corpus (4)

• BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation
• 3D Scene Geometry Estimation from 360${}^{\circ }$ Imagery: A Survey
• Improving 360 Monocular Depth Estimation via Non-local Dense Prediction Transformer and Joint Supervised and Self-supervised Learning
• Monocular Spherical Depth Estimation with Explicitly Connected Weak Layout Cues