Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry

Abstract

Most previous learning-based visual odometry (VO) methods take VO as a pure tracking problem. In contrast, we present a VO framework by incorporating two additional components called Memory and Refining. The Memory component preserves global information by employing an adaptive and efficient selection strategy. The Refining component ameliorates previous results with the contexts stored in the Memory by adopting a spatial-temporal attention mechanism for feature distilling. Experiments on the KITTI and TUM-RGBD benchmark datasets demonstrate that our method outperforms state-of-the-art learning-based methods by a large margin and produces competitive results against classic monocular VO approaches. Especially, our model achieves outstanding performance in challenging scenarios such as texture-less regions and abrupt motions, where classic VO algorithms tend to fail.

Accepted to CVPR2019 (oral presentation)

References in corpus (4)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• FlowNet: Learning Optical Flow with Convolutional Networks

Cited by in corpus (5)

• Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry
• Self-Supervised Deep Visual Odometry with Online Adaptation
• Monocular 3D Object Detection: An Extrinsic Parameter Free Approach
• Self-supervised Visual-LiDAR Odometry with Flip Consistency
• Transformer Guided Geometry Model for Flow-Based Unsupervised Visual Odometry