The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
arXiv:2106.07030 · doi:10.1038/s41467-024-53827-9
Abstract
The capabilities of natural neural systems have inspired new generations of machine learning algorithms as well as neuromorphic very large-scale integrated (VLSI) circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. In this study, we present a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing, implemented on Intel's Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits from the MNIST dataset. To our knowledge, this is the first work to show a Spiking Neural Network (SNN) implementation of the backpropagation algorithm that is fully on-chip, without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.
arXiv comment21 pages, 5 figures, Changes v1->v2: minor changes of text and formatting, correction of total power in supplementary Table III
References in corpus (16)
- Deep Learning in Spiking Neural Networks
- Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks
- Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
- SuperSpike: Supervised learning in multi-layer spiking neural networks
- SLAYER: Spike Layer Error Reassignment in Time
- Recurrent Network Models Of Sequence Generation And Memory
- Rapid online learning and robust recall in a neuromorphic olfactory circuit
- Supervised Learning in Multilayer Spiking Neural Networks
- Mixed-precision deep learning based on computational memory
- Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
- Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks
- Towards Lossless ANN-SNN Conversion under Ultra-Low Latency with Dual-Phase Optimization
- Graded, Dynamically Routable Information Processing with Synfire-Gated Synfire Chains
- Visual Pattern Recognition with on On-chip Learning: towards a Fully Neuromorphic Approach
- A Fokker-Planck approach to graded information propagation in pulse-gated feedforward neuronal networks
- Event-based Backpropagation for Analog Neuromorphic Hardware