BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning

Abstract

Artificial Neural Network (ANN)-based inference on battery-powered devices can be made more energy-efficient by restricting the synaptic weights to be binary, hence eliminating the need to perform multiplications. An alternative, emerging, approach relies on the use of Spiking Neural Networks (SNNs), biologically inspired, dynamic, event-driven models that enhance energy efficiency via the use of binary, sparse, activations. In this paper, an SNN model is introduced that combines the benefits of temporally sparse binary activations and of binary weights. Two learning rules are derived, the first based on the combination of straight-through and surrogate gradient techniques, and the second based on a Bayesian paradigm. Experiments validate the performance loss with respect to full-precision implementations, and demonstrate the advantage of the Bayesian paradigm in terms of accuracy and calibration.

arXiv commentSubmitted

References in corpus (10)

• Categorical Reparameterization with Gumbel-Softmax
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
• Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients
• Training Binary Neural Networks using the Bayesian Learning Rule
• Bayesian Deep Learning via Subnetwork Inference
• Spiking Neural Networks -- Part I: Detecting Spatial Patterns
• Free Energy Minimization: A Unified Framework for Modelling, Inference, Learning,and Optimization