BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

Abstract

We recently proposed the S4NN algorithm, essentially an adaptation of backpropagation to multilayer spiking neural networks that use simple non-leaky integrate-and-fire neurons and a form of temporal coding known as time-to-first-spike coding. With this coding scheme, neurons fire at most once per stimulus, but the firing order carries information. Here, we introduce BS4NN, a modification of S4NN in which the synaptic weights are constrained to be binary (+1 or -1), in order to decrease memory (ideally, one bit per synapse) and computation footprints. This was done using two sets of weights: firstly, real-valued weights, updated by gradient descent, and used in the backward pass of backpropagation, and secondly, their signs, used in the forward pass. Similar strategies have been used to train (non-spiking) binarized neural networks. The main difference is that BS4NN operates in the time domain: spikes are propagated sequentially, and different neurons may reach their threshold at different times, which increases computational power. We validated BS4NN on two popular benchmarks, MNIST and Fashion-MNIST, and obtained reasonable accuracies for this sort of network (97.0% and 87.3% respectively) with a negligible accuracy drop with respect to real-valued weights (0.4% and 0.7%, respectively). We also demonstrated that BS4NN outperforms a simple BNN with the same architectures on those two datasets (by 0.2% and 0.9% respectively), presumably because it leverages the temporal dimension. The source codes of the proposed BS4NN are publicly available at https://github.com/SRKH/BS4NN.

References in corpus (11)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Surrogate Gradient Learning in Spiking Neural Networks
• SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
• Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks
• Efficient supervised learning in networks with binary synapses
• Exploring the Connection Between Binary and Spiking Neural Networks
• Biologically plausible deep learning -- but how far can we go with shallow networks?
• Embedded Binarized Neural Networks
• Technical report: supervised training of convolutional spiking neural networks with PyTorch
• You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy
• A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design

Cited by in corpus (5)

• Spiking neural networks trained via proxy
• An Encoding Framework for Binarized Images using HyperDimensional Computing
• Energy efficiency analysis of Spiking Neural Networks for space applications
• Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks
• A brain basis of dynamical intelligence for AI and computational neuroscience