On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation

Abstract

Inferring the depth of images is a fundamental inverse problem within the field of Computer Vision since depth information is obtained through 2D images, which can be generated from infinite possibilities of observed real scenes. Benefiting from the progress of Convolutional Neural Networks (CNNs) to explore structural features and spatial image information, Single Image Depth Estimation (SIDE) is often highlighted in scopes of scientific and technological innovation, as this concept provides advantages related to its low implementation cost and robustness to environmental conditions. In the context of autonomous vehicles, state-of-the-art CNNs optimize the SIDE task by producing high-quality depth maps, which are essential during the autonomous navigation process in different locations. However, such networks are usually supervised by sparse and noisy depth data, from Light Detection and Ranging (LiDAR) laser scans, and are carried out at high computational cost, requiring high-performance Graphic Processing Units (GPUs). Therefore, we propose a new lightweight and fast supervised CNN architecture combined with novel feature extraction models which are designed for real-world autonomous navigation. We also introduce an efficient surface normals module, jointly with a simple geometric 2.5D loss function, to solve SIDE problems. We also innovate by incorporating multiple Deep Learning techniques, such as the use of densification algorithms and additional semantic, surface normals and depth information to train our framework. The method introduced in this work focuses on robotic applications in indoor and outdoor environments and its results are evaluated on the competitive and publicly available NYU Depth V2 and KITTI Depth datasets.

29 pages, 16 figures. Preprint published in the Elsevier's Robotics and Autonomous Systems journal on November 23, 2020

References in corpus (35)

• Adam: A Method for Stochastic Optimization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
• Deep Learning using Rectified Linear Units (ReLU)
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Squeeze-and-Excitation Networks
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
• From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Confidence Propagation through CNNs for Guided Sparse Depth Regression
• High Quality Monocular Depth Estimation via Transfer Learning
• DeepV2D: Video to Depth with Differentiable Structure from Motion
• BA-Net: Dense Bundle Adjustment Network
• Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
• PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
• MiniNet: An extremely lightweight convolutional neural network for real-time unsupervised monocular depth estimation
• Semantically-Guided Representation Learning for Self-Supervised Monocular Depth
• Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumes
• The BerHu penalty and the grouped effect
• Learning Monocular Depth by Distilling Cross-domain Stereo Networks
• Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
• Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps
• Learning Guided Convolutional Network for Depth Completion
• Attention-based Context Aggregation Network for Monocular Depth Estimation
• Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics
• CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
• DFineNet: Ego-Motion Estimation and Depth Refinement from Sparse, Noisy Depth Input with RGB Guidance
• Structure-Aware Residual Pyramid Network for Monocular Depth Estimation
• Deformable spatial propagation network for depth completion
• Conf-Net: Toward High-Confidence Dense 3D Point-Cloud with Error-Map Prediction
• Structure-Attentioned Memory Network for Monocular Depth Estimation
• Index Network

Cited by in corpus (7)

• NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis
• Boosting Light-Weight Depth Estimation Via Knowledge Distillation
• Co-training for Deep Object Detection: Comparing Single-modal and Multi-modal Approaches
• Towards Real-Time Monocular Depth Estimation for Robotics: A Survey
• Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
• LEAD: LiDAR Extender for Autonomous Driving
• Mirror3D: Depth Refinement for Mirror Surfaces