Statistical mechanics of complex neural systems and high dimensional data

Abstract

Recent experimental advances in neuroscience have opened new vistas into the immense complexity of neuronal networks. This proliferation of data challenges us on two parallel fronts. First, how can we form adequate theoretical frameworks for understanding how dynamical network processes cooperate across widely disparate spatiotemporal scales to solve important computational problems? And second, how can we extract meaningful models of neuronal systems from high dimensional datasets? To aid in these challenges, we give a pedagogical review of a collection of ideas and theoretical methods arising at the intersection of statistical physics, computer science and neurobiology. We introduce the interrelated replica and cavity methods, which originated in statistical physics as powerful ways to quantitatively analyze large highly heterogeneous systems of many interacting degrees of freedom. We also introduce the closely related notion of message passing in graphical models, which originated in computer science as a distributed algorithm capable of solving large inference and optimization problems involving many coupled variables. We then show how both the statistical physics and computer science perspectives can be applied in a wide diversity of contexts to problems arising in theoretical neuroscience and data analysis. Along the way we discuss spin glasses, learning theory, illusions of structure in noise, random matrices, dimensionality reduction, and compressed sensing, all within the unified formalism of the replica method. Moreover, we review recent conceptual connections between message passing in graphical models, and neural computation and learning. Overall, these ideas illustrate how statistical physics and computer science might provide a lens through which we can uncover emergent computational functions buried deep within the dynamical complexities of neuronal networks.

72 pages, 8 figures, iopart.cls, to appear in JSTAT

References in corpus (8)

• Gibbs States and the Set of Solutions of Random Constraint Satisfaction Problems
• Large Deviations of the Maximum Eigenvalue for Wishart and Gaussian Random Matrices
• Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State
• Large Deviations of the Maximum Eigenvalue in Wishart Random Matrices
• Exact distribution of the maximal height of p vicious walkers
• Function Constrains Network Architecture and Dynamics: A Case Study on the Yeast Cell Cycle Boolean Network
• Nonintersecting Brownian excursions
• Theory of spike timing based neural classifiers

Cited by in corpus (22)

• A high-bias, low-variance introduction to Machine Learning for physicists
• Generalisation error in learning with random features and the hidden manifold model
• Coherent Ising machines -- Quantum optics and neural network perspectives
• Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning
• Bayesian Optimal Data Detector for mmWave OFDM System with Low-Resolution ADC
• Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization
• Dimension of activity in random neural networks
• Does the brain behave like a (complex) network? I. Dynamics
• Linear Readout of Object Manifolds
• Mean-field inference methods for neural networks
• Geometric framework to predict structure from function in neural networks
• Statistical Mechanics of High-Dimensional Inference
• Numerical evidence against advantage with quantum fidelity kernels on classical data
• p-Adic Statistical Field Theory and Deep Belief Networks
• Statistical mechanics of low-rank tensor decomposition
• Nonequilibrium physics of brain dynamics
• Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations
• Finite size effects. The averaged eigenvalue density of Wigner random sign real symmetric matrices
• Stochastic Gradient Descent and Anomaly of Variance-flatness Relation in Artificial Neural Networks
• Applications of information geometry to spiking neural network behavior
• Simplified derivations for high-dimensional convex learning problems
• Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis