An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

Abstract

Spiking Neural Networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by hardware implementations that have demonstrated significant energy reductions as compared to conventional Artificial Neural Networks (ANNs). Most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pre-trained ANNs via rate encoding. This paper aims at providing an introduction to SNNs by focusing on a probabilistic signal processing methodology that enables the direct derivation of learning rules leveraging the unique time encoding capabilities of SNNs. To this end, the paper adopts discrete-time probabilistic models for networked spiking neurons, and it derives supervised and unsupervised learning rules from first principles by using variational inference. Examples and open research problems are also provided.

This article is now superseded by arXiv:1910.01059. To appear on IEEE Signal Processing Magazine

References in corpus (11)

• Auto-Encoding Variational Bayes
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• Neural Variational Inference and Learning in Belief Networks
• Surrogate Gradient Learning in Spiking Neural Networks
• Network Plasticity as Bayesian Inference
• Deep Spiking Networks
• Encoding Spike Patterns in Multilayer Spiking Neural Networks
• Stochasticity from function -- why the Bayesian brain may need no noise
• Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients
• Boltzmann machines for time-series
• Low-Power Neuromorphic Hardware for Signal Processing Applications