Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Abstract

Recent work has shown that CNN-based depth and ego-motion estimators can be learned using unlabelled monocular videos. However, the performance is limited by unidentified moving objects that violate the underlying static scene assumption in geometric image reconstruction. More significantly, due to lack of proper constraints, networks output scale-inconsistent results over different samples, i.e., the ego-motion network cannot provide full camera trajectories over a long video sequence because of the per-frame scale ambiguity. This paper tackles these challenges by proposing a geometry consistency loss for scale-consistent predictions and an induced self-discovered mask for handling moving objects and occlusions. Since we do not leverage multi-task learning like recent works, our framework is much simpler and more efficient. Comprehensive evaluation results demonstrate that our depth estimator achieves the state-of-the-art performance on the KITTI dataset. Moreover, we show that our ego-motion network is able to predict a globally scale-consistent camera trajectory for long video sequences, and the resulting visual odometry accuracy is competitive with the recent model that is trained using stereo videos. To the best of our knowledge, this is the first work to show that deep networks trained using unlabelled monocular videos can predict globally scale-consistent camera trajectories over a long video sequence.

arXiv commentAccepted to NeurIPS 2019. Code is available at https://github.com/JiawangBian/SC-SfMLearner-Release

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
• Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos

Cited by in corpus (19)

• Monocular Depth Estimation Based On Deep Learning: An Overview
• Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey
• Unsupervised Monocular Depth Learning in Dynamic Scenes
• Gradient-based Uncertainty for Monocular Depth Estimation
• DF-VO: What Should Be Learnt for Visual Odometry?
• Monocular Depth Estimators: Vulnerabilities and Attacks
• Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods
• On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation
• Toward Better SSIM Loss for Unsupervised Monocular Depth Estimation
• Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications
• Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video
• Instance and Panoptic Segmentation Using Conditional Convolutions
• Information-Theoretic Odometry Learning
• Training Deep SLAM on Single Frames
• Transformers in Unsupervised Structure-from-Motion
• Generalizing to the Open World: Deep Visual Odometry with Online Adaptation
• Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization
• DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Model
• A Front-End for Dense Monocular SLAM using a Learned Outlier Mask Prior