Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Abstract

In this work, we propose a novel framework for unsupervised learning for event cameras that learns motion information from only the event stream. In particular, we propose an input representation of the events in the form of a discretized volume that maintains the temporal distribution of the events, which we pass through a neural network to predict the motion of the events. This motion is used to attempt to remove any motion blur in the event image. We then propose a loss function applied to the motion compensated event image that measures the motion blur in this image. We train two networks with this framework, one to predict optical flow, and one to predict egomotion and depths, and evaluate these networks on the Multi Vehicle Stereo Event Camera dataset, along with qualitative results from a variety of different scenes.

arXiv comment9 pages, 7 figures

References in corpus (9)

• The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• Event-based Moving Object Detection and Tracking
• SfM-Net: Learning of Structure and Motion from Video
• Unsupervised Learning of Depth and Ego-Motion from Video
• Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
• Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
• Recurrent Neural Network for Learning DenseDepth and Ego-Motion from Video
• Realtime Time Synchronized Event-based Stereo

Cited by in corpus (3)

• Focus Is All You Need: Loss Functions For Event-based Vision
• Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
• Motion Equivariant Networks for Event Cameras with the Temporal Normalization Transform