From SLAM to Situational Awareness: Challenges and Survey
arXiv:2110.00273 · doi:10.3390/s23104849
Abstract
The capability of a mobile robot to efficiently and safely perform complex missions is limited by its knowledge of the environment, namely the situation. Advanced reasoning, decision-making, and execution skills enable an intelligent agent to act autonomously in unknown environments. Situational Awareness (SA) is a fundamental capability of humans that has been deeply studied in various fields, such as psychology, military, aerospace, and education. Nevertheless, it has yet to be considered in robotics, which has focused on single compartmentalized concepts such as sensing, spatial perception, sensor fusion, state estimation, and Simultaneous Localization and Mapping (SLAM). Hence, the present research aims to connect the broad multidisciplinary existing knowledge to pave the way for a complete SA system for mobile robotics that we deem paramount for autonomy. To this aim, we define the principal components to structure a robotic SA and their area of competence. Accordingly, this paper investigates each aspect of SA, surveying the state-of-the-art robotics algorithms that cover them, and discusses their current limitations. Remarkably, essential aspects of SA are still immature since the current algorithmic development restricts their performance to only specific environments. Nevertheless, Artificial Intelligence (AI), particularly Deep Learning (DL), has brought new methods to bridge the gap that maintains these fields apart from the deployment to real-world scenarios. Furthermore, an opportunity has been discovered to interconnect the vastly fragmented space of robotic comprehension algorithms through the mechanism of Situational Graph (S-Graph), a generalization of the well-known scene graph. Therefore, we finally shape our vision for the future of robotic Situational Awareness by discussing interesting recent research directions.
22 pages, 8 figures, 5 Tables
References in corpus (69)
- YOLOv4: Optimal Speed and Accuracy of Object Detection
- Rethinking Atrous Convolution for Semantic Image Segmentation
- ORB-SLAM: a Versatile and Accurate Monocular SLAM System
- ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
- ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- Event-based Vision: A Survey
- On-Manifold Preintegration for Real-Time Visual-Inertial Odometry
- Focal Loss for Dense Object Detection
- DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes
- Domain Generalization: A Survey
- DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
- Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review
- Human Motion Trajectory Prediction: A Survey
- Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning
- Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
- The TUM VI Benchmark for Evaluating Visual-Inertial Odometry
- CubeSLAM: Monocular 3D Object SLAM
- Deep Continuous Fusion for Multi-Sensor 3D Object Detection
- F-LOAM: Fast LiDAR Odometry And Mapping
- Event-based Moving Object Detection and Tracking
- Fast-SCNN: Fast Semantic Segmentation Network
- A Comprehensive Survey of Scene Graphs: Generation and Application
- Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes
- A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
- DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
- Implicit Neural Representations with Periodic Activation Functions
- Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
- Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization
- Visual-Inertial Mapping with Non-Linear Factor Recovery
- Continuous-Time Visual-Inertial Odometry for Event Cameras
- RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter
- Social Interactions for Autonomous Driving: A Review and Perspectives
- LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method
- Direct Sparse Mapping
- Visual SLAM: What are the Current Trends and What to Expect?
- VDO-SLAM: A Visual Dynamic Object-aware SLAM System
- LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping
- Safe Local Exploration for Replanning in Cluttered Unknown Environments for Micro-Aerial Vehicles
- SurfelMeshing: Online Surfel-Based Mesh Reconstruction
- A Real-Time Approach for Chance-Constrained Motion Planning with Dynamic Obstacles
- Voxgraph: Globally Consistent, Volumetric Mapping using Signed Distance Function Submaps
- Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
- Scale-Aware Trident Networks for Object Detection
- Keyframe-based Direct Thermal-Inertial Odometry
- Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency
- Loosely-Coupled Semi-Direct Monocular SLAM
- Panoptic Feature Pyramid Networks
- SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud
- Noise modeling and analysis of an IMU-based attitude sensor: improvement of performance by filtering and sensor fusion
- The Oxford Multimotion Dataset: Multiple SE(3) Motions with Ground Truth
- HOG based Fast Human Detection
- SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud
- TensorMask: A Foundation for Dense Object Segmentation
- Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
- Neural Unsigned Distance Fields for Implicit Function Learning
- LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping
- An Online Semantic Mapping System for Extending and Enhancing Visual SLAM
- VIRAL SLAM: Tightly Coupled Camera-IMU-UWB-Lidar SLAM
- Deep Online Correction for Monocular Visual Odometry
- TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs
- Online Temporal Calibration for Monocular Visual-Inertial Systems
- MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network
- LIRO: Tightly Coupled Lidar-Inertia-Ranging Odometry
- Learning to Segment 3D Point Clouds in 2D Image Space
- Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks
Cited by in corpus (7)
- Visual SLAM: What are the Current Trends and What to Expect?
- LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives
- On Digital Twins in Defence: Overview and Applications
- UAV-assisted Visual SLAM Generating Reconstructed 3D Scene Graphs in GPS-denied Environments
- SMapper: A Multi-Modal Data Acquisition Platform for SLAM Benchmarking
- Concept-Guided Exploration: Building Persistent, Actionable Scene Graphs
- Guessing human intentions to avoid dangerous situations in caregiving robots