Predictive Coding: a Theoretical and Experimental Review

Abstract

Predictive coding offers a potentially unifying account of cortical function -- postulating that the core function of the brain is to minimize prediction errors with respect to a generative model of the world. The theory is closely related to the Bayesian brain framework and, over the last two decades, has gained substantial influence in the fields of theoretical and cognitive neuroscience. A large body of research has arisen based on both empirically testing improved and extended theoretical and mathematical models of predictive coding, as well as in evaluating their potential biological plausibility for implementation in the brain and the concrete neurophysiological and psychological predictions made by the theory. Despite this enduring popularity, however, no comprehensive review of predictive coding theory, and especially of recent developments in this field, exists. Here, we provide a comprehensive review both of the core mathematical structure and logic of predictive coding, thus complementing recent tutorials in the literature. We also review a wide range of classic and recent work within the framework, ranging from the neurobiologically realistic microcircuits that could implement predictive coding, to the close relationship between predictive coding and the widely-used backpropagation of error algorithm, as well as surveying the close relationships between predictive coding and modern machine learning techniques.

27/07/21 initial upload; 14/01/22 maths fix; 05/07/22 maths fix; 12/07/22 text fixes

References in corpus (15)

• Learning Transferable Visual Models From Natural Language Supervision
• Language Models are Few-Shot Learners
• Scaling Laws for Neural Language Models
• Jukebox: A Generative Model for Music
• Reinforcement Learning through Active Inference
• Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
• Deep Predictive Coding Networks
• Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
• Relaxing the Constraints on Predictive Coding Models
• Activation Relaxation: A Local Dynamical Approximation to Backpropagation in the Brain
• Neural Kalman Filtering
• Hierarchical Predictive Coding Models in a Deep-Learning Framework
• On Kalman-Bucy filters, linear quadratic control and active inference
• Making Predictive Coding Networks Generative
• Investigating the Scalability and Biological Plausibility of the Activation Relaxation Algorithm

Cited by in corpus (5)

• On the relationship between predictive coding and backpropagation
• Kalman filters as the steady-state solution of gradient descent on variational free energy
• A Mathematical Walkthrough and Discussion of the Free Energy Principle
• Predictive coding, precision and natural gradients
• PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding