Training Spiking Deep Networks for Neuromorphic Hardware

Abstract

We describe a method to train spiking deep networks that can be run using leaky integrate-and-fire (LIF) neurons, achieving state-of-the-art results for spiking LIF networks on five datasets, including the large ImageNet ILSVRC-2012 benchmark. Our method for transforming deep artificial neural networks into spiking networks is scalable and works with a wide range of neural nonlinearities. We achieve these results by softening the neural response function, such that its derivative remains bounded, and by training the network with noise to provide robustness against the variability introduced by spikes. Our analysis shows that implementations of these networks on neuromorphic hardware will be many times more power-efficient than the equivalent non-spiking networks on traditional hardware.

10 pages, 3 figures, 4 tables; the "methods" section of this article draws heavily on arXiv:1510.08829

References in corpus (3)

• Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
• Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems
• Spiking Deep Networks with LIF Neurons

Cited by in corpus (19)

• Deep Learning in Spiking Neural Networks
• Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks
• Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
• Federated Learning with Spiking Neural Networks
• RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network
• Neuromorphic Computing for Content-based Image Retrieval
• Efficient Computation in Adaptive Artificial Spiking Neural Networks
• Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• A Bin Encoding Training of a Spiking Neural Network-based Voice Activity Detection
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks
• Fast and energy-efficient neuromorphic deep learning with first-spike times
• Energy efficiency analysis of Spiking Neural Networks for space applications
• Algorithm and Hardware Design of Discrete-Time Spiking Neural Networks Based on Back Propagation with Binary Activations
• ST-MNIST -- The Spiking Tactile MNIST Neuromorphic Dataset
• Neuromorphic Processing and Sensing: Evolutionary Progression of AI to Spiking
• Minibatch Processing in Spiking Neural Networks
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation
• Spiking Linear Dynamical Systems on Neuromorphic Hardware for Low-Power Brain-Machine Interfaces