Spin glass theory and its new challenge: structured disorder

Abstract

This paper first describes, from a high level viewpoint, the main challenges that had to be solved in order to develop a theory of spin glasses in the last fifty years. It then explains how important inference problems, notably those occurring in machine learning, can be formulated as problems in statistical physics of disordered systems. However, the main questions that we face in the analysis of deep networks require to develop a new chapter of spin glass theory, which will address the challenge of structured data.

arXiv comment17 pages, one figure

References in corpus (10)

• Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model
• The Thermodynamic Limit in Mean Field Spin Glass Models
• Fluctuation-dissipation ratio of a spin glass in the aging regime
• Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
• Shaping the learning landscape in neural networks around wide flat minima
• How to compute the thermodynamics of a glass using a cloned liquid
• Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
• Entropy landscape of solutions in the binary perceptron problem
• Generative diffusion in very large dimensions
• How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model

Cited by in corpus (9)

• Roadmap on machine learning glassy dynamics
• How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model
• Evolutionary accessibility of random and structured fitness landscapes
• Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation
• Classical Annealing of Sherrington-Kirkpatrick Spin Glass Using Suzuki-Kubo Mean-field Ising Dynamics
• Time-translation invariance symmetry breaking hidden by finite-scale singularities
• The Most Dispersed Subset of Random Points in ${\mathbb{R}}^d$
• Robustness of the Random Language Model
• The quantum $p$-spin renormalization group in the large $N$ limit as a benchmark for functional renormalization group