Event-Based Motion Segmentation by Motion Compensation

Abstract

In contrast to traditional cameras, whose pixels have a common exposure time, event-based cameras are novel bio-inspired sensors whose pixels work independently and asynchronously output intensity changes (called "events"), with microsecond resolution. Since events are caused by the apparent motion of objects, event-based cameras sample visual information based on the scene dynamics and are, therefore, a more natural fit than traditional cameras to acquire motion, especially at high speeds, where traditional cameras suffer from motion blur. However, distinguishing between events caused by different moving objects and by the camera's ego-motion is a challenging task. We present the first per-event segmentation method for splitting a scene into independently moving objects. Our method jointly estimates the event-object associations (i.e., segmentation) and the motion parameters of the objects (or the background) by maximization of an objective function, which builds upon recent results on event-based motion-compensation. We provide a thorough evaluation of our method on a public dataset, outperforming the state-of-the-art by as much as 10%. We also show the first quantitative evaluation of a segmentation algorithm for event cameras, yielding around 90% accuracy at 4 pixels relative displacement.

When viewed in Acrobat Reader, several of the figures animate. Video: https://youtu.be/0q6ap_OSBAk

References in corpus (8)

• The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• Event-based Moving Object Detection and Tracking
• Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
• Focus Is All You Need: Loss Functions For Event-based Vision
• Independent Motion Detection with Event-driven Cameras
• Event-Based Features Selection and Tracking from Intertwined Estimation of Velocity and Generative Contours

Cited by in corpus (9)

• Event-based Vision: A Survey
• Focus Is All You Need: Loss Functions For Event-based Vision
• Event Collapse in Contrast Maximization Frameworks
• Globally-Optimal Event Camera Motion Estimation
• Events-to-Video: Bringing Modern Computer Vision to Event Cameras
• Event Probability Mask (EPM) and Event Denoising Convolutional Neural Network (EDnCNN) for Neuromorphic Cameras
• Globally Optimal Contrast Maximisation for Event-based Motion Estimation
• Single Image Optical Flow Estimation with an Event Camera
• Distance Surface for Event-Based Optical Flow