Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey

Abstract

Autonomous systems possess the features of inferring their own state, understanding their surroundings, and performing autonomous navigation. With the applications of learning systems, like deep learning and reinforcement learning, the visual-based self-state estimation, environment perception and navigation capabilities of autonomous systems have been efficiently addressed, and many new learning-based algorithms have surfaced with respect to autonomous visual perception and navigation. In this review, we focus on the applications of learning-based monocular approaches in ego-motion perception, environment perception and navigation in autonomous systems, which is different from previous reviews that discussed traditional methods. First, we delineate the shortcomings of existing classical visual simultaneous localization and mapping (vSLAM) solutions, which demonstrate the necessity to integrate deep learning techniques. Second, we review the visual-based environmental perception and understanding methods based on deep learning, including deep learning-based monocular depth estimation, monocular ego-motion prediction, image enhancement, object detection, semantic segmentation, and their combinations with traditional vSLAM frameworks. Then, we focus on the visual navigation based on learning systems, mainly including reinforcement learning and deep reinforcement learning. Finally, we examine several challenges and promising directions discussed and concluded in related research of learning systems in the era of computer science and robotics.

This paper has been accepted by IEEE TNNLS

References in corpus (18)

• ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
• A Survey of Deep Learning Techniques for Autonomous Driving
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Applications of Deep Learning and Reinforcement Learning to Biological Data
• Visual-Inertial Monocular SLAM with Map Reuse
• Monocular Depth Estimation Based On Deep Learning: An Overview
• DeepFactors: Real-Time Probabilistic Dense Monocular SLAM
• Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning
• UniFuse: Unidirectional Fusion for 360${}^{\circ }$ Panorama Depth Estimation
• Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
• Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency
• Deep Direct Visual Odometry
• One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
• Semi-Dense 3D Semantic Mapping from Monocular SLAM
• Neural Volume Rendering: NeRF And Beyond
• Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Masks
• Self-Supervised Deep Pose Corrections for Robust Visual Odometry
• Sparse Bayesian Inference for Dense Semantic Mapping

Cited by in corpus (5)

• MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer
• Inertial Navigation Meets Deep Learning: A Survey of Current Trends and Future Directions
• Vision-based Learning for Drones: A Survey
• MLANet: Multi-Level Attention Network with Sub-instruction for Continuous Vision-and-Language Navigation
• Inverse RL Scene Dynamics Learning for Nonlinear Predictive Control in Autonomous Vehicles