DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
arXiv:1709.08429 · doi:10.1109/ICRA.2017.7989236
Abstract
This paper studies monocular visual odometry (VO) problem. Most of existing VO algorithms are developed under a standard pipeline including feature extraction, feature matching, motion estimation, local optimisation, etc. Although some of them have demonstrated superior performance, they usually need to be carefully designed and specifically fine-tuned to work well in different environments. Some prior knowledge is also required to recover an absolute scale for monocular VO. This paper presents a novel end-to-end framework for monocular VO by using deep Recurrent Convolutional Neural Networks (RCNNs). Since it is trained and deployed in an end-to-end manner, it infers poses directly from a sequence of raw RGB images (videos) without adopting any module in the conventional VO pipeline. Based on the RCNNs, it not only automatically learns effective feature representation for the VO problem through Convolutional Neural Networks, but also implicitly models sequential dynamics and relations using deep Recurrent Neural Networks. Extensive experiments on the KITTI VO dataset show competitive performance to state-of-the-art methods, verifying that the end-to-end Deep Learning technique can be a viable complement to the traditional VO systems.
Published in 2017 IEEE International Conference on Robotics and Automation (ICRA 2017). Official version at http://ieeexplore.ieee.org/document/7989236/ . Project website and videos at http://senwang.gitlab.io/DeepVO/
References in corpus (2)
Cited by in corpus (143)
- CubeSLAM: Monocular 3D Object SLAM
- DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
- DeepIM: Deep Iterative Matching for 6D Pose Estimation
- Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
- DeepV2D: Video to Depth with Differentiable Structure from Motion
- Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots
- Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning
- LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method
- GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks
- Autonomous Drone Racing: A Survey
- Visual SLAM: What are the Current Trends and What to Expect?
- Deep Learning-based Spacecraft Relative Navigation Methods: A Survey
- A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
- Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas
- A New Wave in Robotics: Survey on Recent mmWave Radar Applications in Robotics
- Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
- LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation
- UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor
- From SLAM to Situational Awareness: Challenges and Survey
- Deep Direct Visual Odometry
- UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning
- Wide-Baseline Relative Camera Pose Estimation with Directional Learning
- Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth
- DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework
- Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
- Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
- Active Neural Localization
- Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
- A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
- D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
- FutureMapping: The Computational Structure of Spatial AI Systems
- Self-Supervised Scale Recovery for Monocular Depth and Egomotion Estimation
- IONet: Learning to Cure the Curse of Drift in Inertial Odometry
- iSLAM: Imperative SLAM
- DeepLO: Geometry-Aware Deep LiDAR Odometry
- DF-VO: What Should Be Learnt for Visual Odometry?
- ViPR: Visual-Odometry-aided Pose Regression for 6DoF Camera Localization
- Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach
- TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimation
- Self-Supervised Deep Pose Corrections for Robust Visual Odometry
- GCNv2: Efficient Correspondence Prediction for Real-Time SLAM
- LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo
- Deep Online Correction for Monocular Visual Odometry
- Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents
- Salient Sparse Visual Odometry With Pose-Only Supervision
- Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information
- Scale-Adaptive Neural Dense Features: Learning via Hierarchical Context Aggregation
- Unsupervised Visual Odometry and Action Integration for PointGoal Navigation in Indoor Environment
- Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry
- Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry
- Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines
- Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy
- MDN-VO: Estimating Visual Odometry with Confidence
- Two Stream Networks for Self-Supervised Ego-Motion Estimation
- Visual Odometry Revisited: What Should Be Learnt?
- milliEgo: Single-chip mmWave Radar Aided Egomotion Estimation via Deep Sensor Fusion
- Local Supports Global: Deep Camera Relocalization with Sequence Enhancement
- Deep Global-Relative Networks for End-to-End 6-DoF Visual Localization and Odometry
- SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
- MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network
- Deep Bayesian ICP Covariance Estimation
- Pose Graph Optimization for Unsupervised Monocular Visual Odometry
- DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features
- Learning to Communicate and Correct Pose Errors
- Multi-Camera Sensor Fusion for Visual Odometry using Deep Uncertainty Estimation
- Self-Supervised Learning of Depth and Ego-motion with Differentiable Bundle Adjustment
- Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
- Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments
- Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
- MVP: Unified Motion and Visual Self-Supervised Learning for Large-Scale Robotic Navigation
- DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
- CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
- Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network
- TartanVO: A Generalizable Learning-based VO
- EMPNet: Neural Localisation and Mapping Using Embedded Memory Points
- Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation
- Learning Multiplicative Interactions with Bayesian Neural Networks for Visual-Inertial Odometry
- WGANVO: Monocular Visual Odometry based on Generative Adversarial Networks
- UL-VIO: Ultra-lightweight Visual-Inertial Odometry with Noise Robust Test-time Adaptation
- Self-Supervised Deep Visual Odometry with Online Adaptation
- PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization
- A comparative evaluation of learned feature descriptors on hybrid monocular visual SLAM methods
- Monocular 3D Object Detection: An Extrinsic Parameter Free Approach
- Deep Model-Based 6D Pose Refinement in RGB
- UnDEMoN 2.0: Improved Depth and Ego Motion Estimation through Deep Image Sampling
- Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video
- PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization
- Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation
- DeepSFM: Structure From Motion Via Deep Bundle Adjustment
- A Survey on 3D LiDAR Localization for Autonomous Vehicles
- OrcVIO: Object residual constrained Visual-Inertial Odometry
- DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints
- Scene Motion Decomposition for Learnable Visual Odometry
- Attentional Separation-and-Aggregation Network for Self-supervised Depth-Pose Learning in Dynamic Scenes
- Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks
- Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
- Line-Circle-Square (LCS): A Multilayered Geometric Filter for Edge-Based Detection
- DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network
- Learning Monocular Visual Odometry through Geometry-Aware Curriculum Learning
- Training Deep SLAM on Single Frames
- Self-supervised Visual-LiDAR Odometry with Flip Consistency
- DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination
- Unsupervised Learning of Camera Pose with Compositional Re-estimation
- Information-Theoretic Odometry Learning
- Semi-Supervised Pipe Video Temporal Defect Interval Localization
- Generalizing to the Open World: Deep Visual Odometry with Online Adaptation
- Robust Consistent Video Depth Estimation
- Virtual Testbed for Monocular Visual Navigation of Small Unmanned Aircraft Systems
- PoseConvGRU: A Monocular Approach for Visual Ego-motion Estimation by Learning
- Scene-Aware Error Modeling of LiDAR/Visual Odometry for Fusion-based Vehicle Localization
- Exploring Self-Attention for Visual Odometry
- Relative Drone-Ground Vehicle Localization using LiDAR and Fisheye Cameras through Direct and Indirect Observations
- GSLAM: A General SLAM Framework and Benchmark
- The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
- DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry
- Experimental Comparison of Visual and Single-Receiver GPS Odometry
- Deep Visual Odometry with Adaptive Memory
- A Multi-spectral Dataset for Evaluating Motion Estimation Systems
- Enhancing Multi-Robot Perception via Learned Data Association
- RAM-VO: Less is more in Visual Odometry
- Incorporating Learnt Local and Global Embeddings into Monocular Visual SLAM
- Sparse2Dense: From direct sparse odometry to dense 3D reconstruction
- Long-Term Vehicle Localization by Recursive Knowledge Distillation
- Motion Consistency Loss for Monocular Visual Odometry with Attention-Based Deep Learning
- Semantics-Driven Unsupervised Learning for Monocular Depth and Ego-Motion Estimation
- VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals
- Improving GNSS Positioning using Neural Network-based Corrections
- Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
- Deep Weakly Supervised Positioning
- Localising In Complex Scenes Using Balanced Adversarial Adaptation
- Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces
- Salient Bundle Adjustment for Visual SLAM
- Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
- Distilling Knowledge From a Deep Pose Regressor Network
- Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approach
- Transformer Guided Geometry Model for Flow-Based Unsupervised Visual Odometry
- LO-Net: Deep Real-time Lidar Odometry
- Use of First and Third Person Views for Deep Intersection Classification
- ATDN vSLAM: An all-through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping
- Guided Feature Selection for Deep Visual Odometry
- Building Intelligent Autonomous Navigation Agents
- Improving drone localisation around wind turbines using monocular model-based tracking
- Deep Visual Odometry Methods for Mobile Robots