Visual Odometry Revisited: What Should Be Learnt?

Abstract

In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue.Some recent deep learning works learn VO in an end-to-end manner but the performance of these deep systems is still not comparable to geometry-based methods. In this work, we revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train two convolutional neural networks (CNNs) for estimating single-view depths and two-view optical flows as intermediate outputs. With the deep predictions, we design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods. More importantly, our system does not suffer from the scale-drift issue being aided by a scale consistent single-view depth CNN. Extensive experiments on KITTI dataset shows the robustness of our system and a detailed ablation study shows the effect of different factors in our system.

arXiv commentICRA2020. Demo video: https://youtu.be/Nl8mFU4SJKY Code: https://github.com/Huangying-Zhan/DF-VO

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
• ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs
• DeepTAM: Deep Tracking and Mapping

Cited by in corpus (5)

• D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
• MVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic Navigation
• Towards Better Generalization: Joint Depth-Pose Learning without PoseNet
• Robot Perception enables Complex Navigation Behavior via Self-Supervised Learning
• Deep Inertial Odometry with Accurate IMU Preintegration