Event Collapse in Contrast Maximization Frameworks

Abstract

Contrast maximization (CMax) is a framework that provides state-of-the-art results on several event-based computer vision tasks, such as ego-motion or optical flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. As prior works have largely ignored the issue or proposed workarounds, it is imperative to analyze this phenomenon in detail. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation based on differential geometry and physics. We experimentally show on publicly available datasets that the proposed metrics mitigate event collapse and do not harm well-posed warps. To the best of our knowledge, regularizers based on the proposed metrics are the only effective solution against event collapse in the experimental settings considered, compared with other methods. We hope that this work inspires further research to tackle more complex warp models.

arXiv comment19 pages, 8 figures, 3 tables

References in corpus (18)

• Event-based Vision: A Survey
• The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
• The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• Event-based Moving Object Detection and Tracking
• A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
• Event-based Stereo Visual Odometry
• Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception
• Continuous-Time Visual-Inertial Odometry for Event Cameras
• Event-Based Motion Segmentation by Motion Compensation
• Secrets of Event-Based Optical Flow
• A compact formula for the derivative of a 3-D rotation in exponential coordinates
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts
• Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks
• Globally-Optimal Contrast Maximisation for Event Cameras
• The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data
• Simultaneous Optical Flow and Segmentation (SOFAS) using Dynamic Vision Sensor

Cited by in corpus (10)

• Secrets of Event-Based Optical Flow
• CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
• Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion
• Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow
• ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras
• A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework
• Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
• Event-based Mosaicing Bundle Adjustment
• Nonlinear Motion-Guided and Spatio-Temporal Aware Network for Unsupervised Event-Based Optical Flow
• FPGA-Based Hardware Architecture for Contrast Maximization in Event-Based Vision