ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
arXiv:2410.09374 · doi:10.1109/TRO.2025.3548523
Abstract
Event-based visual odometry is a specific branch of visual Simultaneous Localization and Mapping (SLAM) techniques, which aims at solving tracking and mapping subproblems (typically in parallel), by exploiting the special working principles of neuromorphic (i.e., event-based) cameras. Due to the motion-dependent nature of event data, explicit data association (i.e., feature matching) under large-baseline view-point changes is difficult to establish, making direct methods a more rational choice. However, state-of-the-art direct methods are limited by the high computational complexity of the mapping sub-problem and the degeneracy of camera pose tracking in certain degrees of freedom (DoF) in rotation. In this paper, we tackle these issues by building an event-based stereo visual-inertial odometry system on top of a direct pipeline. Specifically, to speed up the mapping operation, we propose an efficient strategy for sampling contour points according to the local dynamics of events. The mapping performance is also improved in terms of structure completeness and local smoothness by merging the temporal stereo and static stereo results. To circumvent the degeneracy of camera pose tracking in recovering the pitch and yaw components of general 6-DoF motion, we introduce IMU measurements as motion priors via pre-integration. To this end, a compact back-end is proposed for continuously updating the IMU bias and predicting the linear velocity, enabling an accurate motion prediction for camera pose tracking. The resulting system scales well with modern high-resolution event cameras and leads to better global positioning accuracy in large-scale outdoor environments. Extensive evaluations on five publicly available datasets featuring different resolutions and scenarios justify the superior performance of the proposed system against five state-of-the-art methods.
References in corpus (19)
- ORB-SLAM: a Versatile and Accurate Monocular SLAM System
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
- Event-based Vision: A Survey
- The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
- Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
- A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
- Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
- Continuous-Time Visual-Inertial Odometry for Event Cameras
- Asynchronous, Photometric Feature Tracking using Events and Frames
- Semi-Dense 3D Reconstruction with a Stereo Event Camera
- VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM
- ESVIO: Event-based Stereo Visual Inertial Odometry
- CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
- Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
- Event Collapse in Contrast Maximization Frameworks
- Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow
- EV-Catcher: High-Speed Object Catching Using Low-latency Event-based Neural Networks
- Asynchronous Blob Tracker for Event Cameras
- Event-based Stereo Visual Odometry with Native Temporal Resolution via Continuous-time Gaussian Process Regression