Benchmarking Spiking Neural Network Learning Methods with Varying Locality

Abstract

Spiking Neural Networks (SNNs), providing more realistic neuronal dynamics, have been shown to achieve performance comparable to Artificial Neural Networks (ANNs) in several machine learning tasks. Information is processed as spikes within SNNs in an event-based mechanism that significantly reduces energy consumption. However, training SNNs is challenging due to the non-differentiable nature of the spiking mechanism. Traditional approaches, such as Backpropagation Through Time (BPTT), have shown effectiveness but come with additional computational and memory costs and are biologically implausible. In contrast, recent works propose alternative learning methods with varying degrees of locality, demonstrating success in classification tasks. In this work, we show that these methods share similarities during the training process, while they present a trade-off between biological plausibility and performance. Further, given the implicitly recurrent nature of SNNs, this research investigates the influence of the addition of explicit recurrence to SNNs. We experimentally prove that the addition of explicit recurrent weights enhances the robustness of SNNs. We also investigate the performance of local learning methods under gradient and non-gradient-based adversarial attacks.

References in corpus (15)

• Explaining and Harnessing Adversarial Examples
• Overcoming catastrophic forgetting in neural networks
• Deep Learning in Spiking Neural Networks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• Algorithms for Learning Kernels Based on Centered Alignment
• Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Adaptive Extreme Edge Computing for Wearable Devices
• ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales
• Spikformer: When Spiking Neural Network Meets Transformer
• Is Spiking Secure? A Comparative Study on the Security Vulnerabilities of Spiking and Deep Neural Networks
• Uncovering the Representation of Spiking Neural Networks Trained with Surrogate Gradient
• Adversarially Robust Spiking Neural Networks Through Conversion