NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis

Abstract

Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation. In particular, we propose a novel training method split in three main steps. First, the prediction results of a monocular depth network are warped to an additional view point. Second, we apply an additional image synthesis network, which corrects and improves the quality of the warped RGB image. The output of this network is required to look as similar as possible to the ground-truth view by minimizing the pixel-wise RGB reconstruction error. Third, we reapply the same monocular depth estimation onto the synthesized second view point and ensure that the depth predictions are consistent with the associated ground truth depth. Experimental results prove that our method achieves state-of-the-art or comparable performance on the KITTI and NYU-Depth-v2 datasets with a lightweight and simple vanilla U-Net architecture.

arXiv comment8 pages (main paper), 9 pages (supplementary material), 14 figures, 4 tables

References in corpus (9)

• NeRF++: Analyzing and Improving Neural Radiance Fields
• From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation
• The Replica Dataset: A Digital Replica of Indoor Spaces
• Depth Extraction from Video Using Non-parametric Sampling
• DeepV2D: Video to Depth with Differentiable Structure from Motion
• Detail Preserving Depth Estimation from a Single Image Using Attention Guided Networks
• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
• On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation
• Single image depth estimation by dilated deep residual convolutional neural network and soft-weight-sum inference

Cited by in corpus (2)

• Gradient-based Uncertainty for Monocular Depth Estimation
• Geometric Constraints in Deep Learning Frameworks: A Survey