Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets

Abstract

The way how recurrently connected networks of spiking neurons in the brain acquire powerful information processing capabilities through learning has remained a mystery. This lack of understanding is linked to a lack of learning algorithms for recurrent networks of spiking neurons (RSNNs) that are both functionally powerful and can be implemented by known biological mechanisms. Since RSNNs are simultaneously a primary target for implementations of brain-inspired circuits in neuromorphic hardware, this lack of algorithmic insight also hinders technological progress in that area. The gold standard for learning in recurrent neural networks in machine learning is back-propagation through time (BPTT), which implements stochastic gradient descent with regard to a given loss function. But BPTT is unrealistic from a biological perspective, since it requires a transmission of error signals backwards in time and in space, i.e., from post- to presynaptic neurons. We show that an online merging of locally available information during a computation with suitable top-down learning signals in real-time provides highly capable approximations to BPTT. For tasks where information on errors arises only late during a network computation, we enrich locally available information through feedforward eligibility traces of synapses that can easily be computed in an online manner. The resulting new generation of learning algorithms for recurrent neural networks provides a new understanding of network learning in the brain that can be tested experimentally. In addition, these algorithms provide efficient methods for on-chip training of RSNNs in neuromorphic hardware.

We changed in this version 2 of the paper the name of the new learning algorithms to e-prop, corrected minor errors, added details -- especially for resulting new rules for synaptic plasticity, edited the notation, and included new results for TIMIT

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• LSTM: A Search Space Odyssey
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Supervised Learning in Spiking Neural Networks with FORCE Training
• Deep Rewiring: Training very sparse deep networks
• Measuring abstract reasoning in neural networks
• Task-Driven Convolutional Recurrent Models of the Visual System
• Decoupled Neural Interfaces using Synthetic Gradients
• Understanding Synthetic Gradients and Decoupled Neural Interfaces

Cited by in corpus (18)

• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Adaptive Extreme Edge Computing for Wearable Devices
• Surrogate Gradient Learning in Spiking Neural Networks
• A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning
• Modeling somatic computation with non-neural bioelectric networks
• A bio-inspired bistable recurrent cell allows for long-lasting memory
• Reservoirs learn to learn
• Dimensions of Timescales in Neuromorphic Computing Systems
• Encoding Event-Based Data With a Hybrid SNN Guided Variational Auto-encoder in Neuromorphic Hardware
• Embodied Neuromorphic Vision with Event-Driven Random Backpropagation
• Event-Driven Visual-Tactile Sensing and Learning for Robots
• Efficient LSTM Training with Eligibility Traces
• Convergence and Alignment of Gradient Descent with Random Backpropagation Weights
• Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation
• Cortico-cerebellar networks as decoupling neural interfaces
• A Temporal Neural Network Architecture for Online Learning
• Hybrid Backpropagation Parallel Reservoir Networks
• Universality of Gradient Descent Neural Network Training