TartanVO: A Generalizable Learning-based VO

Abstract

We present the first learning-based visual odometry (VO) model, which generalizes to multiple datasets and real-world scenarios and outperforms geometry-based methods in challenging scenes. We achieve this by leveraging the SLAM dataset TartanAir, which provides a large amount of diverse synthetic data in challenging environments. Furthermore, to make our VO model generalize across datasets, we propose an up-to-scale loss function and incorporate the camera intrinsic parameters into the model. Experiments show that a single model, TartanVO, trained only on synthetic data, without any finetuning, can be generalized to real-world datasets such as KITTI and EuRoC, demonstrating significant advantages over the geometry-based methods on challenging trajectories. Our code is available at https://github.com/castacks/tartanvo.

References in corpus (6)

• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• SfM-Net: Learning of Structure and Motion from Video
• Keyframe-based monocular SLAM: design, survey, and future directions
• BA-Net: Dense Bundle Adjustment Network
• LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo

Cited by in corpus (1)

• DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras