Focus Is All You Need: Loss Functions For Event-based Vision

Abstract

Event cameras are novel vision sensors that output pixel-level brightness changes ("events") instead of traditional video frames. These asynchronous sensors offer several advantages over traditional cameras, such as, high temporal resolution, very high dynamic range, and no motion blur. To unlock the potential of such sensors, motion compensation methods have been recently proposed. We present a collection and taxonomy of twenty two objective functions to analyze event alignment in motion compensation approaches (Fig. 1). We call them Focus Loss Functions since they have strong connections with functions used in traditional shape-from-focus applications. The proposed loss functions allow bringing mature computer vision tools to the realm of event cameras. We compare the accuracy and runtime performance of all loss functions on a publicly available dataset, and conclude that the variance, the gradient and the Laplacian magnitudes are among the best loss functions. The applicability of the loss functions is shown on multiple tasks: rotational motion, depth and optical flow estimation. The proposed focus loss functions allow to unlock the outstanding properties of event cameras.

arXiv comment29 pages, 19 figures, 4 tables

References in corpus (13)

• The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• Event-based Moving Object Detection and Tracking
• A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
• HFirst: A Temporal Approach to Object Recognition
• Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
• Continuous-Time Visual-Inertial Odometry for Event Cameras
• Event-Based Motion Segmentation by Motion Compensation
• Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
• Optimal Piecewise Linear Function Approximation for GPU-based Applications
• Optimal polygonal L1 linearization and fast interpolation of nonlinear systems
• Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Cited by in corpus (6)

• Event-based Vision: A Survey
• Event-based Stereo Visual Odometry
• Event-Based Motion Segmentation by Motion Compensation
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning
• Globally Optimal Contrast Maximisation for Event-based Motion Estimation
• Distance Surface for Event-Based Optical Flow