BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

Abstract

We recently proposed the S4NN algorithm, essentially an adaptation of backpropagation to multilayer spiking neural networks that use simple non-leaky integrate-and-fire neurons and a form of temporal coding known as time-to-first-spike coding. With this coding scheme, neurons fire at most once per stimulus, but the firing order carries information. Here, we introduce BS4NN, a modification of S4NN in which the synaptic weights are constrained to be binary (+1 or -1), in order to decrease memory (ideally, one bit per synapse) and computation footprints. This was done using two sets of weights: firstly, real-valued weights, updated by gradient descent, and used in the backward pass of backpropagation, and secondly, their signs, used in the forward pass. Similar strategies have been used to train (non-spiking) binarized neural networks. The main difference is that BS4NN operates in the time domain: spikes are propagated sequentially, and different neurons may reach their threshold at different times, which increases computational power. We validated BS4NN on two popular benchmarks, MNIST and Fashion-MNIST, and obtained reasonable accuracies for this sort of network (97.0% and 87.3% respectively) with a negligible accuracy drop with respect to real-valued weights (0.4% and 0.7%, respectively). We also demonstrated that BS4NN outperforms a simple BNN with the same architectures on those two datasets (by 0.2% and 0.9% respectively), presumably because it leverages the temporal dimension. The source codes of the proposed BS4NN are publicly available at https://github.com/SRKH/BS4NN.

References in corpus (29)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Deep Learning in Spiking Neural Networks
• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• STDP-based spiking deep convolutional neural networks for object recognition
• SuperSpike: Supervised learning in multi-layer spiking neural networks
• SLAYER: Spike Layer Error Reassignment in Time
• HFirst: A Temporal Approach to Object Recognition
• Neuromorphic Deep Learning Machines
• Spiking Deep Networks with LIF Neurons
• First-spike based visual categorization using reward-modulated STDP
• S4NN: temporal backpropagation for spiking neural networks with one spike per neuron
• A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• Surrogate Gradient Learning in Spiking Neural Networks
• Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks
• Efficient supervised learning in networks with binary synapses
• Exploring the Connection Between Binary and Spiking Neural Networks
• Biologically plausible deep learning -- but how far can we go with shallow networks?
• Embedded Binarized Neural Networks
• Optimal localist and distributed coding of spatiotemporal spike patterns through STDP and coincidence detection
• Technical report: supervised training of convolutional spiking neural networks with PyTorch
• Deep Convolutional Spiking Neural Networks for Image Classification
• You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy
• A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design

Cited by in corpus (5)

• Spiking neural networks trained via proxy
• An Encoding Framework for Binarized Images using HyperDimensional Computing
• Energy efficiency analysis of Spiking Neural Networks for space applications
• Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks
• A brain basis of dynamical intelligence for AI and computational neuroscience