On-Manifold Preintegration for Real-Time Visual-Inertial Odometry
arXiv:1512.02363 · doi:10.1109/TRO.2016.2597321
Abstract
Current approaches for visual-inertial odometry (VIO) are able to attain highly accurate state estimation via nonlinear optimization. However, real-time optimization quickly becomes infeasible as the trajectory grows over time, this problem is further emphasized by the fact that inertial measurements come at high rate, hence leading to fast growth of the number of variables in the optimization. In this paper, we address this issue by preintegrating inertial measurements between selected keyframes into single relative motion constraints. Our first contribution is a \emph{preintegration theory} that properly addresses the manifold structure of the rotation group. We formally discuss the generative measurement model as well as the nature of the rotation noise and derive the expression for the \emph{maximum a posteriori} state estimator. Our theoretical development enables the computation of all necessary Jacobians for the optimization and a-posteriori bias correction in analytic form. The second contribution is to show that the preintegrated IMU model can be seamlessly integrated into a visual-inertial pipeline under the unifying framework of factor graphs. This enables the application of incremental-smoothing algorithms and the use of a \emph{structureless} model for visual measurements, which avoids optimizing over the 3D points, further accelerating the computation. We perform an extensive evaluation of our monocular \VIO pipeline on real and simulated datasets. The results confirm that our modelling effort leads to accurate state estimation in real-time, outperforming state-of-the-art approaches.
20 pages, 24 figures, accepted for publication in IEEE Transactions on Robotics (TRO) 2016
Cited by in corpus (150)
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
- ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- Visual-Inertial Monocular SLAM with Map Reuse
- Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
- Using Inertial Sensors for Position and Orientation Estimation
- A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
- Continuous-Time Visual-Inertial Odometry for Event Cameras
- VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots
- A micro Lie theory for state estimation in robotics
- Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
- Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry
- SVIn2: An Underwater SLAM System using Sonar, Visual, Inertial, and Depth Sensor
- Event-aided Direct Sparse Odometry
- A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
- LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping
- Robust Legged Robot State Estimation Using Factor Graph Optimization
- Closed-form Preintegration Methods for Graph-based Visual-Inertial Navigation
- AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System
- From SLAM to Situational Awareness: Challenges and Survey
- RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments
- OxIOD: The Dataset for Deep Inertial Odometry
- Edge-Assisted Multi-Robot Visual-Inertial SLAM with Efficient Communication
- Contact-Aided Invariant Extended Kalman Filtering for Legged Robot State Estimation
- Hybrid Contact Preintegration for Visual-Inertial-Contact State Estimation Using Factor Graphs
- Range-Visual-Inertial Odometry: Scale Observability Without Excitation
- MILIOM: Tightly Coupled Multi-Input Lidar-Inertia Odometry and Mapping
- Deep IMU Bias Inference for Robust Visual-Inertial Odometry with Factor Graphs
- The Geometry of Navigation Problems
- CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
- Robust Inertial-aided Underwater Localization based on Imaging Sonar Keyframes
- Online Motion Planning based on Nonlinear Model Predictive Control with Non-Euclidean Rotation Groups
- HybVIO: Pushing the Limits of Real-time Visual-inertial Odometry
- GLIM: 3D Range-Inertial Localization and Mapping with GPU-Accelerated Scan Matching Factors
- PIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
- Preintegrated Velocity Bias Estimation to Overcome Contact Nonlinearities in Legged Robot Odometry
- GNSS/Multi-Sensor Fusion Using Continuous-Time Factor Graph Optimization for Robust Localization
- Feature-based visual odometry prior for real-time semi-dense stereo SLAM
- Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry
- Experimental Evaluation of Visual-Inertial Odometry Systems for Arable Farming
- A Fast and Robust Algorithm for Orientation Estimation using Inertial Sensors
- A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
- D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
- Rolling-Shutter Modelling for Direct Visual-Inertial Odometry
- LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping
- Factor Graph Fusion of Raw GNSS Sensing with IMU and Lidar for Precise Robot Localization without a Base Station
- VIRAL SLAM: Tightly Coupled Camera-IMU-UWB-Lidar SLAM
- FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter
- Minimization of GNSS-Denied Inertial Navigation Errors for Fixed Wing Autonomous Unmanned Air Vehicles
- Multi-Robot Relative Pose Estimation and IMU Preintegration Using Passive UWB Transceivers
- Guaranteed Performance Nonlinear Observer for Simultaneous Localization and Mapping
- Probabilistic Spatial Distribution Prior Based Attentional Keypoints Matching Network
- Navigating by Touch: Haptic Monte Carlo Localization via Geometric Sensing and Terrain Classification
- Probabilistic Degeneracy Detection for Point-to-Plane Error Minimization
- GNSS-stereo-inertial SLAM for arable farming
- Visual-Inertial Navigation: A Concise Review
- Factor Graph-Based Smoothing Without Matrix Inversion for Highly Precise Localization
- Observability-aware online multi-lidar extrinsic calibration
- ConFusion: Sensor Fusion for Complex Robotic Systems using Nonlinear Optimization
- The Complex-Step Derivative Approximation on Matrix Lie Groups
- Low Latency Visual Inertial Odometry with On-Sensor Accelerated Optical Flow for Resource-Constrained UAVs
- Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review
- Know What You Don't Know: Consistency in Sliding Window Filtering with Unobservable States Applied to Visual-Inertial SLAM (Extended Version)
- Tightly-Coupled LiDAR-IMU-Wheel Odometry with an Online Neural Kinematic Model Learning via Factor Graph Optimization
- A Factor-Graph Approach for Optimization Problems with Dynamics Constraints
- Versatile LiDAR-Inertial Odometry With SE (2) Constraints for Ground Vehicles
- A flexible framework for accurate LiDAR odometry, map manipulation, and localization
- A Radio-Inertial Localization and Tracking System with BLE Beacons Prior Maps
- FAST-LIO2: Fast Direct LiDAR-inertial Odometry
- An Improved Multi-State Constraint Kalman Filter for Visual-Inertial Odometry
- RoadMap: A Light-Weight Semantic Map for Visual Localization towards Autonomous Driving
- SLAM2REF: Advancing Long-Term Mapping with 3D LiDAR and Reference Map Integration for Precise 6-DoF Trajectory Estimation and Map Extension
- AVP-SLAM: Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot
- IBISCape: A Simulated Benchmark for multi-modal SLAM Systems Evaluation in Large-scale Dynamic Environments
- An Invariant-EKF VINS Algorithm for Improving Consistency
- Pose Estimation for Ground Robots: On Manifold Representation, Integration, Re-Parameterization, and Optimization
- A Review of Visual Odometry Methods and Its Applications for Autonomous Driving
- Image Based Camera Localization: an Overview
- G-Loc: Tightly-coupled Graph Localization with Prior Topo-metric Information
- Events-to-Video: Bringing Modern Computer Vision to Event Cameras
- GNSS-Denied Semi Direct Visual Navigation for Autonomous UAVs Aided by PI-Inspired Inertial Priors
- Robust Single Rotation Averaging
- VIRAL-Fusion: A Visual-Inertial-Ranging-Lidar Sensor Fusion Approach
- Enhancing State Estimator for Autonomous Racing : Leveraging Multi-modal System and Managing Computing Resources
- Tightly-Coupled LiDAR-IMU-Leg Odometry with Online Learned Leg Kinematics Incorporating Foot Tactile Information
- Accurate IMU Preintegration Using Switched Linear Systems For Autonomous Systems
- BoundMPC: Cartesian path following with error bounds based on model predictive control in the joint space
- Long Distance GNSS-Denied Visual Inertial Navigation for Autonomous Fixed Wing Unmanned Air Vehicles: SO(3) Manifold Filter based on Virtual Vision Sensor
- M-LIO: Multi-lidar, multi-IMU odometry with sensor dropout tolerance
- Trifo-VIO: Robust and Efficient Stereo Visual Inertial Odometry using Points and Lines
- Laser map aided visual inertial localization in changing environment
- LiDAR and Inertial Fusion for Pose Estimation by Non-linear Optimization
- UL-VIO: Ultra-lightweight Visual-Inertial Odometry with Noise Robust Test-time Adaptation
- Equivariant IMU Preintegration with Biases: a Galilean Group Approach
- Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems
- Adaptive Continuous Visual Odometry from RGB-D Images
- Continuous Direct Sparse Visual Odometry from RGB-D Images
- Online Initialization and Extrinsic Spatial-Temporal Calibration for Monocular Visual-Inertial Odometry
- A Unified Method for Solving Inverse, Forward, and Hybrid Manipulator Dynamics using Factor Graphs
- From Underground Mines to Offices: A Versatile and Robust Framework for Range-Inertial SLAM
- Haptic Sequential Monte Carlo Localization for Quadrupedal Locomotion in Vision-Denied Scenarios
- An Energy-Efficient Quad-Camera Visual System for Autonomous Machines on FPGA Platform
- Incorporating Control Inputs in Continuous-Time Gaussian Process State Estimation for Robotics
- Resilient Terrain Navigation with a 5 DOF Metal Detector Drone
- DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints
- Communication constrained cloud-based long-term visual localization in real time
- VIR-SLAM: Visual, Inertial, and Ranging SLAM for single and multi-robot systems
- The Unified Mathematical Framework for IMU Preintegration in Inertial-Aided Navigation System
- A Lightweight and Accurate Localization Algorithm Using Multiple Inertial Measurement Units
- Visual Inertial Odometry using Focal Plane Binary Features (BIT-VIO)
- Real-time Indoor Scene Reconstruction with RGBD and Inertia Input
- IMU Data Processing For Inertial Aided Navigation: A Recurrent Neural Network Based Approach
- Fast Extrinsic Calibration for Multiple Inertial Measurement Units in Visual-Inertial System
- 3DMNDT:3D multi-view registration method based on the normal distributions transform
- Enhancing Visual Inertial SLAM with Magnetic Measurements
- VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation
- A Right Invariant Extended Kalman Filter for Object based SLAM
- Asynchronous Event-Inertial Odometry using a Unified Gaussian Process Regression Framework
- LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices
- 3D Freehand Ultrasound using Visual Inertial and Deep Inertial Odometry for Measuring Patellar Tracking
- Hyperspherical Dirac Mixture Reapproximation
- Geometric Structure Aided Visual Inertial Localization
- A Versatile Keyframe-Based Structureless Filter for Visual Inertial Odometry
- SE-SLAM: Semi-Dense Structured Edge-Based Monocular SLAM
- Low-Cost GPS-Aided LiDAR State Estimation and Map Building
- A Robust Pavement Mapping System Based on Normal-Constrained Stereo Visual Odometry
- Globally optimal consensus maximization for robust visual inertial localization in point and line map
- SPINS: Structure Priors aided Inertial Navigation System
- High Speed Tracking With A Fourier Domain Kernelized Correlation Filter
- Exploring the Accuracy Potential of IMU Preintegration in Factor Graph Optimization
- GSLAM: A General SLAM Framework and Benchmark
- Quadrotor Control on with SLAM Integration
- Learned Uncertainty Calibration for Visual Inertial Localization
- Invariant Smoothing for Localization: Including the IMU Biases
- On the Existence of Linear Observed Systems on Manifolds with Connection
- GEVO: Memory-Efficient Monocular Visual Odometry Using Gaussians
- MIMC-VINS: A Versatile and Resilient Multi-IMU Multi-Camera Visual-Inertial Navigation System
- Piecewise Linear De-skewing for LiDAR Inertial Odometry
- State Estimation in Visual Inertial Autonomous Helicopter Landing Using Optimisation on Manifold
- Improving the Iterative Closest Point Algorithm using Lie Algebra
- RIO: Rotation-equivariance supervised learning of robust inertial odometry
- Avoiding Degeneracy for Monocular Visual SLAM with Point and Line Features
- Deep Inertial Odometry with Accurate IMU Preintegration
- DVIO: Depth aided visual inertial odometry for RGBD sensors
- Direct Visual-Inertial Odometry with Semi-Dense Mapping
- Invariant Smoothing on Lie Groups
- 2Fast-2Lamaa: Large-Scale Lidar-Inertial Localization and Mapping with Continuous Distance Fields
- Convergence and Consistency Analysis for A 3D Invariant-EKF SLAM
- Invariant Cubature Kalman Filter for Monocular Visual Inertial Odometry with Line Features
- Extrinsic Calibration and Verification of Multiple Non-overlapping Field of View Lidar Sensors