DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data

Abstract

We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation, thus preserving accurate shapes of a scene. Previous methods that predict metric depth often work well only for a specific scene. In contrast, learning relative depth (information of being closer or further) can enjoy better generalization, with the price of failing to recover the accurate geometric shape of the scene. In this work, we propose a dataset and methods to tackle this dilemma, aiming to predict accurate depth up to an affine transformation with good generalization to diverse scenes. First we construct a large-scale and diverse dataset, termed Diverse Scene Depth dataset (DiverseDepth), which has a broad range of scenes and foreground contents. Compared with previous learning objectives, i.e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes. Furthermore, in order to train the model on the complex dataset effectively, we propose a multi-curriculum learning method. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting, demonstrating the excellent generalization capacity of the learned model to diverse scenes. The reconstructed point clouds with the predicted depth show that our method can recover high-quality 3D shapes. Code and dataset are available at: https://tinyurl.com/DiverseDepth

Fixed typos

References in corpus (5)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• High Quality Monocular Depth Estimation via Transfer Learning
• Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks
• On The Power of Curriculum Learning in Training Deep Networks
• A Large RGB-D Dataset for Semi-supervised Monocular Depth Estimation

Cited by in corpus (8)

• Conditional Convolutions for Instance Segmentation
• Boosting Light-Weight Depth Estimation Via Knowledge Distillation
• Online Mutual Adaptation of Deep Depth Prediction and Visual SLAM
• Learning to Recover 3D Scene Shape from a Single Image
• Instance and Panoptic Segmentation Using Conditional Convolutions
• Towards Domain-agnostic Depth Completion
• Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
• Predicting 3D shapes, masks, and properties of materials, liquids, and objects inside transparent containers, using the TransProteus CGI dataset