Deep Spiking Networks

Abstract

We introduce an algorithm to do backpropagation on a spiking network. Our network is "spiking" in the sense that our neurons accumulate their activation into a potential over time, and only send out a signal (a "spike") when this potential crosses a threshold and the neuron is reset. Neurons only update their states when receiving signals from other neurons. Total computation of the network thus scales with the number of spikes caused by an input rather than network size. We show that the spiking Multi-Layer Perceptron behaves identically, during both prediction and training, to a conventional deep network of rectified-linear units, in the limiting case where we run the spiking network for a long time. We apply this architecture to a conventional classification problem (MNIST) and achieve performance very close to that of a conventional Multi-Layer Perceptron with the same architecture. Our network is a natural architecture for learning based on streaming event-based data, and is a stepping stone towards using spiking neural networks to learn efficiently on streaming data.

8 pages main paper + 1 page reference + 7 pages Appendix

References in corpus (2)

• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Spiking Deep Networks with LIF Neurons

Cited by in corpus (21)

• Deep Learning in Spiking Neural Networks
• Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
• First-spike based visual categorization using reward-modulated STDP
• Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
• Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks
• Efficient Spiking Neural Networks with Logarithmic Temporal Coding
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes
• Temporally Efficient Deep Learning with Spikes
• Sigma Delta Quantized Networks
• Deep Spiking Neural Network with Spike Count based Learning Rule
• Direct CMOS Implementation of Neuromorphic Temporal Neural Networks for Sensory Processing
• Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks
• Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization
• Minibatch Processing in Spiking Neural Networks
• An Efficient Approach to Boosting Performance of Deep Spiking Network Training
• SpinAPS: A High-Performance Spintronic Accelerator for Probabilistic Spiking Neural Networks
• Bifurcation Spiking Neural Network
• An Introduction to Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications
• Better Safe than Sorry: Evidence Accumulation Allows for Safe Reinforcement Learning
• In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor