Summary statistics of learning link changing neural representations to behavior

Abstract

How can we make sense of large-scale recordings of neural activity across learning? Theories of neural network learning with their origins in statistical physics offer a potential answer: for a given task, there are often a small set of summary statistics that are sufficient to predict performance as the network learns. Here, we review recent advances in how summary statistics can be used to build theoretical understanding of neural network learning. We then argue for how this perspective can inform the analysis of neural data, enabling better understanding of learning in biological and artificial neural networks.

arXiv comment12 pages, 2 figures

References in corpus (19)

• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Statistical physics of inference: Thresholds and algorithms
• Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
• Modelling the influence of data structure on learning in neural networks: the hidden manifold model
• Getting aligned on representational alignment
• Asymptotics of representation learning in finite Bayesian neural networks
• Depth induces scale-averaging in overparameterized linear Bayesian neural networks
• Coding schemes in neural networks learning classification tasks
• On neural network kernels and the storage capacity problem
• Scaling and renormalization in high-dimensional regression
• The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks
• Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network
• Bayes-optimal Learning of Deep Random Networks of Extensive-width
• A Spectral Theory of Neural Prediction and Alignment
• Loss Dynamics of Temporal Difference Reinforcement Learning
• Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
• What Representational Similarity Measures Imply about Decodable Information
• A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator