Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
arXiv:1807.02570
Abstract
Monocular visual odometry approaches that purely rely on geometric cues are prone to scale drift and require sufficient motion parallax in successive frames for motion estimation and 3D reconstruction. In this paper, we propose to leverage deep monocular depth prediction to overcome limitations of geometry-based monocular visual odometry. To this end, we incorporate deep depth predictions into Direct Sparse Odometry (DSO) as direct virtual stereo measurements. For depth prediction, we design a novel deep network that refines predicted depth from a single image in a two-stage process. We train our network in a semi-supervised way on photoconsistency in stereo images and on consistency with accurate sparse depth reconstructions from Stereo DSO. Our deep predictions excel state-of-the-art approaches for monocular depth on the KITTI benchmark. Moreover, our Deep Virtual Stereo Odometry clearly exceeds previous monocular and deep learning based methods in accuracy. It even achieves comparable performance to the state-of-the-art stereo methods, while only relying on a single camera.
arXiv commentTo appear in ECCV 2018, Munich. 17 pages including references, 7 figures, 4 tables. Supplementary material: https://vision.in.tum.de/members/yangn
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- Self-Supervised Learning for Stereo Matching with Self-Improving Ability
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction
- CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
- Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras
- Unsupervised Monocular Depth Estimation with Left-Right Consistency