Automotive Object Detection via Learning Sparse Events by Spiking Neurons
arXiv:2307.12900 · doi:10.1109/TCDS.2024.3410371
Abstract
Event-based sensors, distinguished by their high temporal resolution of 1 and a dynamic range of 120 , stand out as ideal tools for deployment in fast-paced settings like vehicles and drones. Traditional object detection techniques that utilize Artificial Neural Networks (ANNs) face challenges due to the sparse and asynchronous nature of the events these sensors capture. In contrast, Spiking Neural Networks (SNNs) offer a promising alternative, providing a temporal representation that is inherently aligned with event-based data. This paper explores the unique membrane potential dynamics of SNNs and their ability to modulate sparse events. We introduce an innovative spike-triggered adaptive threshold mechanism designed for stable training. Building on these insights, we present a specialized spiking feature pyramid network (SpikeFPN) optimized for automotive event-based object detection. Comprehensive evaluations demonstrate that SpikeFPN surpasses both traditional SNNs and advanced ANNs enhanced with attention mechanisms. Evidently, SpikeFPN achieves a mean Average Precision (mAP) of 0.477 on the GEN1 Automotive Detection (GAD) benchmark dataset, marking significant increases over the selected SNN baselines. Moreover, the efficient design of SpikeFPN ensures robust performance while optimizing computational resources, attributed to its innate sparse computation capabilities. Source codes are publicly accessible at https://github.com/EMI-Group/spikefpn.
IEEE Transactions on Cognitive and Developmental Systems
References in corpus (22)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- SSD: Single Shot MultiBox Detector
- Decoupled Weight Decay Regularization
- SuperSpike: Supervised learning in multi-layer spiking neural networks
- Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
- EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
- Learning Spatial Fusion for Single-Shot Object Detection
- SLAYER: Spike Layer Error Reassignment in Time
- HFirst: A Temporal Approach to Object Recognition
- Deep Residual Learning in Spiking Neural Networks
- Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks
- Learning to Detect Objects with a 1 Megapixel Event Camera
- Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks
- Depth Quality Aware Salient Object Detection
- Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting
- Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks
- A Large Scale Event-based Detection Dataset for Automotive
- Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks
- Asynchronous Spatial Image Convolutions for Event Cameras
- Accelerated physical emulation of Bayesian inference in spiking neural networks
- Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
- Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network