Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks

Abstract

Spiking neural networks combine analog computation with event-based communication using discrete spikes. While the impressive advances of deep learning are enabled by training non-spiking artificial neural networks using the backpropagation algorithm, applying this algorithm to spiking networks was previously hindered by the existence of discrete spike events and discontinuities. For the first time, this work derives the backpropagation algorithm for a continuous-time spiking neural network and a general loss function by applying the adjoint method together with the proper partial derivative jumps, allowing for backpropagation through discrete spike events without approximations. This algorithm, EventProp, backpropagates errors at spike times in order to compute the exact gradient in an event-based, temporally and spatially sparse fashion. We use gradients computed via EventProp to train networks on the Yin-Yang and MNIST datasets using either a spike time or voltage based loss function and report competitive performance. Our work supports the rigorous study of gradient-based learning algorithms in spiking neural networks and provides insights toward their implementation in novel brain-inspired hardware.

References in corpus (1)

• A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)

Cited by in corpus (17)

• Spiking neural networks trained via proxy
• Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks
• Smooth Exact Gradient Descent Learning in Spiking Neural Networks
• Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systems
• Learning through structure: towards deep neuromorphic knowledge graph embeddings
• Neuro-symbolic computing with spiking neural networks
• Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors
• FeNN: A RISC-V vector processor for Spiking Neural Network acceleration
• Backpropagation through space, time, and the brain
• Spiking representation learning for associative memories
• Relational representation learning with spike trains
• SpikeCV: Open a Continuous Computer Vision Era
• Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions
• A flexible framework for structural plasticity in GPU-accelerated sparse spiking neural networks
• Exploring Spiking Neural Networks for Binary Classification in Multivariate Time Series at the Edge
• Ultrafast neural sampling with spiking nanolasers
• Three factor delay learning rules for spiking neural networks