Three factor delay learning rules for spiking neural networks

Abstract

Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.

7 pages, 5 figures

References in corpus (10)

• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
• ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales
• Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks
• Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation
• Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks
• S7: Selective and Simplified State Space Layers for Sequence Modeling