Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth

Abstract

Deep learning-based, single-view depth estimation methods have recently shown highly promising results. However, such methods ignore one of the most important features for determining depth in the human vision system, which is motion. We propose a learning-based, multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view image reprojection and forward-backward flow-consistency losses. Our model can be trained in a supervised or even unsupervised mode. It is designed for depth and visual odometry estimation from video where the input frames are temporally correlated. However, it also generalizes to single-view depth estimation. Our method produces superior results to the state-of-the-art approaches for single-view and multi-view learning-based depth estimation on the KITTI driving dataset.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• SfM-Net: Learning of Structure and Motion from Video
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration

Cited by in corpus (18)

• Monocular Depth Estimation Based On Deep Learning: An Overview
• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
• Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency
• Deep Direct Visual Odometry
• Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumes
• LiDAR Odometry Methodologies for Autonomous Driving: A Survey
• Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
• VOLDOR-SLAM: For the Times When Feature-Based or Direct Methods Are Not Good Enough
• Monocular 3D Object Detection: An Extrinsic Parameter Free Approach
• Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
• Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment
• Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks
• Is Depth Really Necessary for Salient Object Detection?
• Monocular Depth Prediction through Continuous 3D Loss
• VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals
• Transformer Guided Geometry Model for Flow-Based Unsupervised Visual Odometry
• Deep 3D Pan via adaptive "t-shaped" convolutions with global and local adaptive dilations
• Semantics-Driven Unsupervised Learning for Monocular Depth and Ego-Motion Estimation