S4NN: temporal backpropagation for spiking neural networks with one spike per neuron

Abstract

We propose a new supervised learning rule for multilayer spiking neural networks (SNNs) that use a form of temporal coding known as rank-order-coding. With this coding scheme, all neurons fire exactly one spike per stimulus, but the firing order carries information. In particular, in the readout layer, the first neuron to fire determines the class of the stimulus. We derive a new learning rule for this sort of network, named S4NN, akin to traditional error backpropagation, yet based on latencies. We show how approximated error gradients can be computed backward in a feedforward network with any number of layers. This approach reaches state-of-the-art performance with supervised multi fully-connected layer SNNs: test accuracy of 97.4% for the MNIST dataset, and 99.2% for the Caltech Face/Motorbike dataset. Yet, the neuron model that we use, non-leaky integrate-and-fire, is much simpler than the one used in all previous works. The source codes of the proposed S4NN are publicly available at https://github.com/SRKH/S4NN.

References in corpus (12)

• Deep Learning in Neural Networks: An Overview
• Deep Learning in Spiking Neural Networks
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• STDP-based spiking deep convolutional neural networks for object recognition
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• HFirst: A Temporal Approach to Object Recognition
• Neuromorphic Deep Learning Machines
• First-spike based visual categorization using reward-modulated STDP
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• Biologically plausible deep learning -- but how far can we go with shallow networks?

Cited by in corpus (19)

• Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks
• Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• Deep SCNN-based Real-time Object Detection for Self-driving Vehicles Using LiDAR Temporal Data
• An Analytical Estimation of Spiking Neural Networks Energy Efficiency
• Fluctuation-driven initialization for spiking neural network training
• Hardware Implementation of Spiking Neural Networks Using Time-To-First-Spike Encoding
• Integration of Leaky-Integrate-and-Fire-Neurons in Deep Learning Architectures
• Spiking neural networks trained via proxy
• Smooth Exact Gradient Descent Learning in Spiking Neural Networks
• Paired Competing Neurons Improving STDP Supervised Local Learning In Spiking Neural Networks
• Efficient spike encoding algorithms for neuromorphic speech recognition
• Fast and energy-efficient neuromorphic deep learning with first-spike times
• SpikE: spike-based embeddings for multi-relational graph data
• Beyond Weights: Deep learning in Spiking Neural Networks with pure synaptic-delay training
• Neuro-symbolic computing with spiking neural networks
• Energy efficiency analysis of Spiking Neural Networks for space applications
• Relational representation learning with spike trains
• A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware