Summary statistics of learning link changing neural representations to behavior

Abstract

How can we make sense of large-scale recordings of neural activity across learning? Theories of neural network learning with their origins in statistical physics offer a potential answer: for a given task, there are often a small set of summary statistics that are sufficient to predict performance as the network learns. Here, we review recent advances in how summary statistics can be used to build theoretical understanding of neural network learning. We then argue for how this perspective can inform the analysis of neural data, enabling better understanding of learning in biological and artificial neural networks.

12 pages, 2 figures

References in corpus (14)

• Getting aligned on representational alignment
• Asymptotics of representation learning in finite Bayesian neural networks
• Depth induces scale-averaging in overparameterized linear Bayesian neural networks
• Coding schemes in neural networks learning classification tasks
• On neural network kernels and the storage capacity problem
• Scaling and renormalization in high-dimensional regression
• The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks
• Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network
• Bayes-optimal Learning of Deep Random Networks of Extensive-width
• A Spectral Theory of Neural Prediction and Alignment
• Loss Dynamics of Temporal Difference Reinforcement Learning
• Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
• What Representational Similarity Measures Imply about Decodable Information
• A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator