Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
arXiv:1804.01310 · doi:10.1109/CVPR.2018.00568
Abstract
Event cameras are bio-inspired vision sensors that naturally capture the dynamics of a scene, filtering out redundant information. This paper presents a deep neural network approach that unlocks the potential of event cameras on a challenging motion-estimation task: prediction of a vehicle's steering angle. To make the best out of this sensor-algorithm combination, we adapt state-of-the-art convolutional architectures to the output of event sensors and extensively evaluate the performance of our approach on a publicly available large scale event-camera dataset (~1000 km). We present qualitative and quantitative explanations of why event cameras allow robust steering prediction even in cases where traditional cameras fail, e.g. challenging illumination conditions and fast motion. Finally, we demonstrate the advantages of leveraging transfer learning from traditional to event-based vision, and show that our approach outperforms state-of-the-art algorithms based on standard cameras.
arXiv comment9 pages, 8 figures, 6 tables. Video: https://youtu.be/_r_bsjkJTHA
References in corpus (2)
Cited by in corpus (34)
- Event-based Vision: A Survey
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies
- A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
- Continuous-Time Visual-Inertial Odometry for Event Cameras
- Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks
- Event-Based Fusion for Motion Deblurring with Cross-modal Attention
- Asynchronous, Photometric Feature Tracking using Events and Frames
- How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
- Event-based visual place recognition with ensembles of temporal windows
- Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation
- Motion Robust High-Speed Light-Weighted Object Detection With Event Camera
- SODFormer: Streaming Object Detection with Transformer Using Events and Frames
- Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking
- LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor
- Improving the Transferability of Adversarial Examples with Arbitrary Style Transfer
- Hypergraph-based Multi-View Action Recognition using Event Cameras
- Neuromorphic Imaging with Joint Image Deblurring and Event Denoising
- Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams
- EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision
- A Composable Dynamic Sparse Dataflow Architecture for Efficient Event-based Vision Processing on FPGA
- A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
- Low-power, Continuous Remote Behavioral Localization with Event Cameras
- Detection of Binary Square Fiducial Markers Using an Event Camera
- Automotive Object Detection via Learning Sparse Events by Spiking Neurons
- Neuromorphic Imaging and Classification with Graph Learning
- REITS: Reflective Surface for Intelligent Transportation Systems
- An N-Point Linear Solver for Line and Motion Estimation with Event Cameras
- Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras
- An Asynchronous Intensity Representation for Framed and Event Video Sources
- Efficient Implementation of a Multi-Layer Gradient-Free Online-Trainable Spiking Neural Network on FPGA
- Event-based Detectors for Laser Guide Star Tip-Tilt Sensing
- Scalable Event-Based Video Streaming for Machines with MoQ
- FPGA-Based Hardware Architecture for Contrast Maximization in Event-Based Vision
- Nonlinear Motion-Guided and Spatio-Temporal Aware Network for Unsupervised Event-Based Optical Flow