DF-VO: What Should Be Learnt for Visual Odometry?

Abstract

Multi-view geometry-based methods dominate the last few decades in monocular Visual Odometry for their superior performance, while they have been vulnerable to dynamic and low-texture scenes. More importantly, monocular methods suffer from scale-drift issue, i.e., errors accumulate over time. Recent studies show that deep neural networks can learn scene depths and relative camera in a self-supervised manner without acquiring ground truth labels. More surprisingly, they show that the well-trained networks enable scale-consistent predictions over long videos, while the accuracy is still inferior to traditional methods because of ignoring geometric information. Building on top of recent progress in computer vision, we design a simple yet robust VO system by integrating multi-view geometry and deep learning on Depth and optical Flow, namely DF-VO. In this work, a) we propose a method to carefully sample high-quality correspondences from deep flows and recover accurate camera poses with a geometric module; b) we address the scale-drift issue by aligning geometrically triangulated depths to the scale-consistent deep depths, where the dynamic scenes are taken into account. Comprehensive ablation studies show the effectiveness of the proposed method, and extensive evaluation results show the state-of-the-art performance of our system, e.g., Ours (1.652%) v.s. ORB-SLAM (3.247%}) in terms of translation error in KITTI Odometry benchmark. Source code is publicly available at: \href{https://github.com/Huangying-Zhan/DF-VO}{DF-VO}.

extended version of ICRA-2020 paper (Visual Odometry Revisited: What Should Be Learnt?)

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video
• VDO-SLAM: A Visual Dynamic Object-aware SLAM System
• UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning
• ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs
• Self-Supervised 3D Keypoint Learning for Ego-motion Estimation
• DeepTAM: Deep Tracking and Mapping

Cited by in corpus (2)

• Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach
• Dense Prediction Transformer for Scale Estimation in Monocular Visual Odometry