ESVIO: Event-based Stereo Visual Inertial Odometry

Abstract

Event cameras that asynchronously output low-latency event streams provide great opportunities for state estimation under challenging situations. Despite event-based visual odometry having been extensively studied in recent years, most of them are based on monocular and few research on stereo event vision. In this paper, we present ESVIO, the first event-based stereo visual-inertial odometry, which leverages the complementary advantages of event streams, standard images and inertial measurements. Our proposed pipeline achieves temporal tracking and instantaneous matching between consecutive stereo event streams, thereby obtaining robust state estimation. In addition, the motion compensation method is designed to emphasize the edge of scenes by warping each event to reference moments with IMU and ESVIO back-end. We validate that both ESIO (purely event-based) and ESVIO (event with image-aided) have superior performance compared with other image-based and event-based baseline methods on public and self-collected datasets. Furthermore, we use our pipeline to perform onboard quadrotor flights under low-light environments. A real-world large-scale experiment is also conducted to demonstrate long-term effectiveness. We highlight that this work is a real-time, accurate system that is aimed at robust state estimation under challenging environments.

References in corpus (8)

• ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM
• Event-based Vision: A Survey
• The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
• Event-based Stereo Visual Odometry
• Continuous-Time Visual-Inertial Odometry for Event Cameras
• VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM

Cited by in corpus (7)

• Autonomous Drone Racing: A Survey
• ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
• Event-based Stereo Depth Estimation: A Survey
• Event-based Stereo Visual Odometry with Native Temporal Resolution via Continuous-time Gaussian Process Regression
• DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System
• On the Benefits of Visual Stabilization for Frame- and Event-based Perception
• AsynEIO: Asynchronous Monocular Event-Inertial Odometry Using Gaussian Process Regression