Learning to solve the credit assignment problem

Abstract

Backpropagation is driving today's artificial neural networks (ANNs). However, despite extensive research, it remains unclear if the brain implements this algorithm. Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic alternative: neurons can randomly introduce change, and use unspecific feedback signals to observe their effect on the cost and thus approximate their gradient. However, the convergence rate of such learning scales poorly with the number of involved neurons. Here we propose a hybrid learning approach. Each neuron uses an RL-type strategy to learn how to approximate the gradients that backpropagation would provide. We provide proof that our approach converges to the true gradient for certain classes of networks. In both feedforward and convolutional networks, we empirically show that our approach learns to approximate the gradient, and can match or the performance of exact gradient-based learning. Learning feedback weights provides a biologically plausible mechanism of achieving good performance, without the need for precise, pre-specified learning rules.

arXiv comment18 pages; 4 figures. (ICLR 2020 version)

References in corpus (12)

• Adding Gradient Noise Improves Learning for Very Deep Networks
• Generalized Denoising Auto-Encoders as Generative Models
• Direct Feedback Alignment Provides Learning in Deep Neural Networks
• Gradient learning in spiking neural networks by dynamic perturbation of conductances
• Random feedback weights support learning in deep neural networks
• Decoupled Neural Interfaces using Synthetic Gradients
• Deep Learning without Weight Transport
• Feedback alignment in deep convolutional networks
• Biologically-plausible learning algorithms can scale to large datasets
• Understanding Synthetic Gradients and Decoupled Neural Interfaces
• Conducting Credit Assignment by Aligning Local Representations
• Differentiable plasticity: training plastic neural networks with backpropagation

Cited by in corpus (7)

• A Theoretical Framework for Target Propagation
• On Training Implicit Models
• Cortical credit assignment by Hebbian, neuromodulatory and inhibitory plasticity
• Biological credit assignment through dynamic inversion of feedforward networks
• Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures
• Credit Assignment Through Broadcasting a Global Error Vector
• Front Contribution instead of Back Propagation