FutureMapping: The Computational Structure of Spatial AI Systems

Abstract

We discuss and predict the evolution of Simultaneous Localisation and Mapping (SLAM) into a general geometric and semantic `Spatial AI' perception capability for intelligent embodied devices. A big gap remains between the visual perception performance that devices such as augmented reality eyewear or comsumer robots will require and what is possible within the constraints imposed by real products. Co-design of algorithms, processors and sensors will be needed. We explore the computational structure of current and future Spatial AI algorithms and consider this within the landscape of ongoing hardware developments.

References in corpus (11)

• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
• Unsupervised Learning of Depth and Ego-Motion from Video
• A Berkeley View of Systems Challenges for AI
• Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
• CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction
• Spatial Transformer Networks
• SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks
• SLAMBench2: Multi-Objective Head-to-Head Benchmarking for Visual SLAM
• gvnn: Neural Network Library for Geometric Computer Vision
• Algorithmic Performance-Accuracy Trade-off in 3D Vision Applications Using HyperMapper

Cited by in corpus (16)

• A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Rearrangement: A Challenge for Embodied AI
• Self-Improving Visual Odometry
• A visual introduction to Gaussian Belief Propagation
• Localization and Mapping using Instance-specific Mesh Models
• SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representations
• MORPH-DSLAM: Model Order Reduction for PHysics-based Deformable SLAM
• Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping
• FutureMapping 2: Gaussian Belief Propagation for Spatial AI
• Building Proactive Voice Assistants: When and How (not) to Interact
• Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems
• Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection
• Online Learning of a Probabilistic and Adaptive Scene Representation
• Advances in Inference and Representation for Simultaneous Localization and Mapping
• SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition