CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization

Abstract

Event cameras are bio-inspired visual sensors that capture pixel-wise intensity changes and output asynchronous event streams. They show great potential over conventional cameras to handle challenging scenarios in robotics and computer vision, such as high-speed and high dynamic range. This paper considers the problem of rotational motion estimation using event cameras. Several event-based rotation estimation methods have been developed in the past decade, but their performance has not been evaluated and compared under unified criteria yet. In addition, these prior works do not consider a global refinement step. To this end, we conduct a systematic study of this problem with two objectives in mind: summarizing previous works and presenting our own solution. First, we compare prior works both theoretically and experimentally. Second, we propose the first event-based rotation-only bundle adjustment (BA) approach. We formulate it leveraging the state-of-the-art Contrast Maximization (CMax) framework, which is principled and avoids the need to convert events into frames. Third, we use the proposed BA to build CMax-SLAM, the first event-based rotation-only SLAM system comprising a front-end and a back-end. Our BA is able to run both offline (trajectory smoothing) and online (CMax-SLAM back-end). To demonstrate the performance and versatility of our method, we present comprehensive experiments on synthetic and real-world datasets, including indoor, outdoor and space scenarios. We discuss the pitfalls of real-world evaluation and propose a proxy for the reprojection error as the figure of merit to evaluate event-based rotation BA methods. We release the source code and novel data sequences to benefit the community. We hope this work leads to a better understanding and fosters further research on event-based ego-motion estimation. Project page: https://github.com/tub-rip/cmax_slam

arXiv comment22 pages, 20 figures, 8 tables. https://github.com/tub-rip/cmax_slam

References in corpus (17)

• Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
• Event-based Vision: A Survey
• On-Manifold Preintegration for Real-Time Visual-Inertial Odometry
• The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
• Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
• Continuous-Time Visual-Inertial Odometry for Event Cameras
• Secrets of Event-Based Optical Flow
• Efficient Derivative Computation for Cumulative B-Splines on Lie Groups
• Globally-Optimal Contrast Maximisation for Event Cameras
• Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
• The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data
• Event Collapse in Contrast Maximization Frameworks
• Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow
• Globally-Optimal Event Camera Motion Estimation
• A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework

Cited by in corpus (8)

• ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
• Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
• ES-PTAM: Event-based Stereo Parallel Tracking and Mapping
• Event-guided Low-light Video Semantic Segmentation
• MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
• Event-based Photometric Bundle Adjustment
• Event-based Mosaicing Bundle Adjustment
• FPGA-Based Hardware Architecture for Contrast Maximization in Event-Based Vision