Event-aided Direct Sparse Odometry

Abstract

We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time between frames. The method formulates a direct probabilistic approach of observed brightness increments. Per-pixel brightness increments are predicted using a sparse number of selected 3D points and are compared to the events via the brightness increment error to estimate camera motion. The method recovers a semi-dense 3D map using photometric bundle adjustment. EDS is the first method to perform 6-DOF VO using events and frames with a direct approach. By design, it overcomes the problem of changing appearance in indirect methods. We also show that, for a target error performance, EDS can work at lower frame rates than state-of-the-art frame-based VO solutions. This opens the door to low-power motion-tracking applications where frames are sparingly triggered "on demand" and our method tracks the motion in between. We release code and datasets to the public.

16 pages, 14 Figures, Page: https://rpg.ifi.uzh.ch/eds

References in corpus (1)

• Learning High-Speed Flight in the Wild

Cited by in corpus (11)

• Autonomous Drone Racing: A Survey
• Event-based Background-Oriented Schlieren
• Event-based Stereo Depth Estimation: A Survey
• A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
• Data-Driven Feature Tracking for Event Cameras With and Without Frames
• DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System
• An N-Point Linear Solver for Line and Motion Estimation with Event Cameras
• ES-PTAM: Event-based Stereo Parallel Tracking and Mapping
• MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
• Event-based Photometric Bundle Adjustment
• Event-based Mosaicing Bundle Adjustment