Secrets of Event-Based Optical Flow

Abstract

Event cameras respond to scene dynamics and offer advantages to estimate motion. Following recent image-based deep-learning achievements, optical flow estimation methods for event cameras have rushed to combine those image-based methods with event data. However, it requires several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate optical flow from events alone. We investigate key elements: how to design the objective function to prevent overfitting, how to warp events to deal better with occlusions, and how to improve convergence with multi-scale raw events. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark, and is competitive on the DSEC benchmark. Moreover, our method allows us to expose the issues of the ground truth flow in those benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. Our code is available at https://github.com/tub-rip/event_based_optical_flow

23 pages, 11 figures, 7 tables, https://github.com/tub-rip/event_based_optical_flow

References in corpus (3)

• Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks
• Globally-Optimal Contrast Maximisation for Event Cameras
• Event Collapse in Contrast Maximization Frameworks

Cited by in corpus (19)

• Optical flow estimation from event-based cameras and spiking neural networks
• CMax-SLAM: Event-based Rotational-Motion Bundle Adjustment and SLAM System using Contrast Maximization
• Event-based Background-Oriented Schlieren
• Event Collapse in Contrast Maximization Frameworks
• Neuromorphic Perception and Navigation for Mobile Robots: A Review
• Lightweight Event-based Optical Flow Estimation via Iterative Deblurring
• Neuromorphic Imaging with Joint Image Deblurring and Event Denoising
• Fast Event-based Optical Flow Estimation by Triplet Matching
• A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework
• Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
• Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras
• Neuromorphic Shack-Hartmann wave normal sensing
• Event Encryption: Rethinking Privacy Exposure for Neuromorphic Imaging
• On the Benefits of Visual Stabilization for Frame- and Event-based Perception
• Combined Physics and Event Camera Simulator for Slip Detection
• Event-Free Moving Object Segmentation from Moving Ego Vehicle
• Event2Audio: Event-Based Optical Vibration Sensing
• Non-Uniform Exposure Imaging via Neuromorphic Shutter Control
• Nonlinear Motion-Guided and Spatio-Temporal Aware Network for Unsupervised Event-Based Optical Flow