End-to-end Learning of Object Motion Estimation from Retinal Events for Event-based Object Tracking

Abstract

Event cameras, which are asynchronous bio-inspired vision sensors, have shown great potential in computer vision and artificial intelligence. However, the application of event cameras to object-level motion estimation or tracking is still in its infancy. The main idea behind this work is to propose a novel deep neural network to learn and regress a parametric object-level motion/transform model for event-based object tracking. To achieve this goal, we propose a synchronous Time-Surface with Linear Time Decay (TSLTD) representation, which effectively encodes the spatio-temporal information of asynchronous retinal events into TSLTD frames with clear motion patterns. We feed the sequence of TSLTD frames to a novel Retinal Motion Regression Network (RMRNet) to perform an end-to-end 5-DoF object motion regression. Our method is compared with state-of-the-art object tracking methods, that are based on conventional cameras or event cameras. The experimental results show the superiority of our method in handling various challenging environments such as fast motion and low illumination conditions.

arXiv comment9 pages, 3 figures

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Event-based Moving Object Detection and Tracking
• Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
• Deep Learning Models of the Retinal Response to Natural Scenes
• SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks
• Robust Estimation of Similarity Transformation for Visual Object Tracking
• Real-time clustering and multi-target tracking using event-based sensors

Cited by in corpus (9)

• Globally-Optimal Contrast Maximisation for Event Cameras
• Asynchronous Blob Tracker for Event Cameras
• VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features
• Object Tracking by Jointly Exploiting Frame and Event Domain
• SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks
• Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera
• Event Data Association via Robust Model Fitting for Event-based Object Tracking
• Multi-domain Collaborative Feature Representation for Robust Visual Object Tracking