Smooth Exact Gradient Descent Learning in Spiking Neural Networks

Abstract

Gradient descent prevails in artificial neural network training, but seems inept for spiking neural networks as small parameter changes can cause sudden, disruptive (dis-)appearances of spikes. Here, we demonstrate exact gradient descent based on continuously changing spiking dynamics. These are generated by neuron models whose spikes vanish and appear at the end of a trial, where it cannot influence subsequent dynamics. This also enables gradient-based spike addition and removal. We illustrate our scheme with various tasks and setups, including recurrent and deep, initially silent networks.

References in corpus (5)

• Array Programming with NumPy
• Neural network models and deep learning - a primer for biologists
• Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State
• Stable Irregular Dynamics in Complex Neural Networks
• Designing Complex Networks

Cited by in corpus (1)

• A ghost mechanism: An analytical model of abrupt learning in recurrent networks