A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence

Abstract

Deep learning based localization and mapping has recently attracted significant attention. Instead of creating hand-designed algorithms through exploitation of physical models or geometric theories, deep learning based solutions provide an alternative to solve the problem in a data-driven way. Benefiting from ever-increasing volumes of data and computational power, these methods are fast evolving into a new area that offers accurate and robust systems to track motion and estimate scenes and their structure for real-world applications. In this work, we provide a comprehensive survey, and propose a new taxonomy for localization and mapping using deep learning. We also discuss the limitations of current models, and indicate possible future directions. A wide range of topics are covered, from learning odometry estimation, mapping, to global localization and simultaneous localization and mapping (SLAM). We revisit the problem of perceiving self-motion and scene understanding with on-board sensors, and show how to solve it by integrating these modules into a prospective spatial machine intelligence system (SMIS). It is our hope that this work can connect emerging works from robotics, computer vision and machine learning communities, and serve as a guide for future researchers to apply deep learning to tackle localization and mapping problems.

26 pages, 10 figures. Project website: https://github.com/changhao-chen/deep-learning-localization-mapping

References in corpus (16)

• Language Models are Few-Shot Learners
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
• Learning to Navigate in Complex Environments
• R2D2: Repeatable and Reliable Detector and Descriptor
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Convolutional Neural Network-based Place Recognition
• UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor
• Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth
• ASLFeat: Learning Local Features of Accurate Shape and Localization
• Geometry-Aware Neural Rendering
• D2D: Keypoint Extraction with Describe to Detect Approach
• DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints
• Neural Multisensory Scene Inference
• AtLoc: Attention Guided Camera Localization