Generalization of Equilibrium Propagation to Vector Field Dynamics

Abstract

The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists. Two major reasons are that neurons would need to send two different types of signal in the forward and backward phases, and that pairs of neurons would need to communicate through symmetric bidirectional connections. We present a simple two-phase learning procedure for fixed point recurrent networks that addresses both these issues. In our model, neurons perform leaky integration and synaptic weights are updated through a local mechanism. Our learning method generalizes Equilibrium Propagation to vector field dynamics, relaxing the requirement of an energy function. As a consequence of this generalization, the algorithm does not compute the true gradient of the objective function, but rather approximates it at a precision which is proven to be directly related to the degree of symmetry of the feedforward and feedback weights. We show experimentally that our algorithm optimizes the objective function.

References in corpus (1)

• Towards deep learning with spiking neurons in energy based models with contrastive Hebbian plasticity

Cited by in corpus (10)

• Training End-to-End Analog Neural Networks with Equilibrium Propagation
• Predictive Coding: a Theoretical and Experimental Review
• Equilibrium Propagation with Continual Weight Updates
• Relaxing the Constraints on Predictive Coding Models
• Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the Brain
• A deep learning theory for neural networks grounded in physics
• Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
• Equilibrium Propagation for Complete Directed Neural Networks
• Investigating the Scalability and Biological Plausibility of the Activation Relaxation Algorithm
• Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation