D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry

Abstract

We propose D3VO as a novel framework for monocular visual odometry that exploits deep networks on three levels -- deep depth, pose and uncertainty estimation. We first propose a novel self-supervised monocular depth estimation network trained on stereo videos without any external supervision. In particular, it aligns the training image pairs into similar lighting condition with predictive brightness transformation parameters. Besides, we model the photometric uncertainties of pixels on the input images, which improves the depth estimation accuracy and provides a learned weighting function for the photometric residuals in direct (feature-less) visual odometry. Evaluation results show that the proposed network outperforms state-of-the-art self-supervised depth estimation networks. D3VO tightly incorporates the predicted depth, pose and uncertainty into a direct visual odometry method to boost both the front-end tracking as well as the back-end non-linear optimization. We evaluate D3VO in terms of monocular visual odometry on both the KITTI odometry benchmark and the EuRoC MAV dataset.The results show that D3VO outperforms state-of-the-art traditional monocular VO methods by a large margin. It also achieves comparable results to state-of-the-art stereo/LiDAR odometry on KITTI and to the state-of-the-art visual-inertial odometry on EuRoC MAV, while using only a single camera.

arXiv commentCVPR 2020

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• On-Manifold Preintegration for Real-Time Visual-Inertial Odometry
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Visual-Inertial Monocular SLAM with Map Reuse
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
• Unsupervised Learning of Depth and Ego-Motion from Video
• CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
• UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning
• Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras
• Unsupervised Monocular Depth Estimation with Left-Right Consistency
• Self-Improving Visual Odometry
• Visual Odometry Revisited: What Should Be Learnt?

Cited by in corpus (7)

• CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation
• Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
• PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry
• Self-supervised Visual-LiDAR Odometry with Flip Consistency
• Generalizing to the Open World: Deep Visual Odometry with Online Adaptation
• CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations
• ATDN vSLAM: An all-through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping