Event-based Moving Object Detection and Tracking

Abstract

Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis. The unique properties encompassed in the readings of such sensors provide high temporal resolution, superior sensitivity to light and low latency. These properties provide the grounds to estimate motion extremely reliably in the most sophisticated scenarios but they come at a price - modern event-based vision sensors have extremely low resolution and produce a lot of noise. Moreover, the asynchronous nature of the event stream calls for novel algorithms. This paper presents a new, efficient approach to object tracking with asynchronous cameras. We present a novel event stream representation which enables us to utilize information about the dynamic (temporal) component of the event stream, and not only the spatial component, at every moment of time. This is done by approximating the 3D geometry of the event stream with a parametric model; as a result, the algorithm is capable of producing the motion-compensated event stream (effectively approximating egomotion), and without using any form of external sensors in extremely low-light and noisy conditions without any form of feature tracking or explicit optical flow computation. We demonstrate our framework on the task of independent motion detection and tracking, where we use the temporal model inconsistencies to locate differently moving objects in challenging situations of very fast motion.

8 pages, 8 figures

References in corpus (5)

• Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
• Independent Motion Detection with Event-driven Cameras
• Real-Time Panoramic Tracking for Event Cameras

Cited by in corpus (49)

• Event-based Vision: A Survey
• Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception
• Event-Based Motion Segmentation by Motion Compensation
• Focus Is All You Need: Loss Functions For Event-based Vision
• Secrets of Event-Based Optical Flow
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts
• End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking
• Event-based visual place recognition with ensembles of temporal windows
• From SLAM to Situational Awareness: Challenges and Survey
• Free as a Bird: Event-based Dynamic Sense-and-Avoid for Ornithopter Robot Flight
• Moving Object Detection for Event-based vision using Graph Spectral Clustering
• Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking
• Inceptive Event Time-Surfaces for Object Classification Using Neuromorphic Cameras
• Globally-Optimal Contrast Maximisation for Event Cameras
• The Spatio-Temporal Poisson Point Process: A Simple Model for the Alignment of Event Camera Data
• Event Collapse in Contrast Maximization Frameworks
• The GRIFFIN Perception Dataset: Bridging the Gap Between Flapping-Wing Flight and Robotic Perception
• Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
• Neuromorphic Perception and Navigation for Mobile Robots: A Review
• Globally-Optimal Event Camera Motion Estimation
• Neutron-Induced, Single-Event Effects on Neuromorphic Event-based Vision Sensor: A First Step Towards Space Applications
• VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows
• Asynchronous Blob Tracker for Event Cameras
• Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy
• Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing
• FAITH: Fast iterative half-plane focus of expansion estimation using event-based optic flow
• e-TLD: Event-based Framework for Dynamic Object Tracking
• Detection of Binary Square Fiducial Markers Using an Event Camera
• Moving Object Detection for Event-based Vision using k-means Clustering
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning
• Event-based tracking of human hands
• Tracking 6-DoF Object Motion from Events and Frames
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
• ASAP: Adaptive Transmission Scheme for Online Processing of Event-based Algorithms
• Event-based Camera Simulation using Monte Carlo Path Tracing with Adaptive Denoising
• Object Tracking by Jointly Exploiting Frame and Event Domain
• Event Probability Mask (EPM) and Event Denoising Convolutional Neural Network (EDnCNN) for Neuromorphic Cameras
• SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks
• MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice
• Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion
• Realtime Time Synchronized Event-based Stereo
• Learning Event-Based Motion Deblurring
• A Joint Intensity-Neuromorphic Event Imaging System for Resource Constrained Devices
• Event Data Association via Robust Model Fitting for Event-based Object Tracking
• Real-Time Grasping Strategies Using Event Camera
• Multi-domain Collaborative Feature Representation for Robust Visual Object Tracking
• Sequence-SOD: Bio-inspired Sequence-aware Spiking ObjectDetection for Event Cameras
• Nonlinear Motion-Guided and Spatio-Temporal Aware Network for Unsupervised Event-Based Optical Flow
• EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream