Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias

Abstract

Monocular depth predictors are typically trained on large-scale training sets which are naturally biased w.r.t the distribution of camera poses. As a result, trained predictors fail to make reliable depth predictions for testing examples captured under uncommon camera poses. To address this issue, we propose two novel techniques that exploit the camera pose during training and prediction. First, we introduce a simple perspective-aware data augmentation that synthesizes new training examples with more diverse views by perturbing the existing ones in a geometrically consistent manner. Second, we propose a conditional model that exploits the per-image camera pose as prior knowledge by encoding it as a part of the input. We show that jointly applying the two methods improves depth prediction on images captured under uncommon and even never-before-seen camera poses. We show that our methods improve performance when applied to a range of different predictor architectures. Lastly, we show that explicitly encoding the camera pose distribution improves the generalization performance of a synthetically trained depth predictor when evaluated on real images.

arXiv commentAccepted at CVPR2021, Oral

References in corpus (7)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• Learning Depth from Single Images with Deep Neural Network Embedding Focal Length
• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset