Disordered Systems Insights on Computational Hardness

Abstract

In this review article, we discuss connections between the physics of disordered systems, phase transitions in inference problems, and computational hardness. We introduce two models representing the behavior of glassy systems, the spiked tensor model and the generalized linear model. We discuss the random (non-planted) versions of these problems as prototypical optimization problems, as well as the planted versions (with a hidden solution) as prototypical problems in statistical inference and learning. Based on ideas from physics, many of these problems have transitions where they are believed to jump from easy (solvable in polynomial time) to hard (requiring exponential time). We discuss several emerging ideas in theoretical computer science and statistics that provide rigorous evidence for hardness by proving that large classes of algorithms fail in the conjectured hard regime. This includes the overlap gap property, a particular mathematization of clustering or dynamical symmetry-breaking, which can be used to show that many algorithms that are local or robust to changes in their input fail. We also discuss the sum-of-squares hierarchy, which places bounds on proofs or algorithms that use low-degree polynomials such as standard spectral methods and semidefinite relaxations, including the Sherrington-Kirkpatrick model. Throughout the manuscript, we present connections to the physics of disordered systems and associated replica symmetry breaking properties.

arXiv comment42 pages

References in corpus (26)

• The dynamics of message passing on dense graphs, with applications to compressed sensing
• Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
• Spectral redemption: clustering sparse networks
• Statistical physics of inference: Thresholds and algorithms
• The Thermodynamic Limit in Mean Field Spin Glass Models
• Clustering of solutions in the random satisfiability problem
• Universality in polytope phase transitions and message passing algorithms
• Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models
• Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices
• Rigorous decimation-based construction of ground pure states for spin glass models on random lattices
• Hiding Quiet Solutions in Random Constraint Satisfaction Problems
• A ferromagnet with a glass transition
• The Overlap Gap Property: a Geometric Barrier to Optimizing over Random Structures
• Sum-of-squares proofs and the quest toward optimal algorithms
• Constrained Low-rank Matrix Estimation: Phase Transitions, Approximate Message Passing and Applications
• Statistical and computational phase transitions in spiked tensor estimation
• Suboptimality of local algorithms for a class of max-cut problems
• Typology of phase transitions in Bayesian inference problems
• Storage capacity in symmetric binary perceptrons
• Quiet Planting in the Locked Constraint Satisfaction Problems
• Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
• Approximate Survey Propagation for Statistical Inference
• On the Bit Complexity of Sum-of-Squares Proofs
• The estimation error of general first order methods
• Recovery thresholds in the sparse planted matching problem
• A Tight Degree 4 Sum-of-Squares Lower Bound for the Sherrington-Kirkpatrick Hamiltonian

Cited by in corpus (22)

• Is stochastic thermodynamics the key to understanding the energy costs of computation?
• Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
• Energy landscapes of combinatorial optimization in Ising machines
• Low-depth Clifford circuits approximately solve MaxCut
• Toward a Theory of Phase Transitions in Quantum Control Landscapes
• Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials
• Bounds on the ground state energy of quantum $p$-spin Hamiltonians
• On the free energy of vector spin glasses with non-convex interactions
• Relaxation Control of Open Quantum Systems
• Statistical mechanics of the maximum-average submatrix problem
• Counting and Hardness-of-Finding Fixed Points in Cellular Automata on Random Graphs
• Combinatorial NLTS From the Overlap Gap Property
• Neural-prior stochastic block model
• The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions
• The Overlap Gap Property limits limit swapping in the QAOA
• Topological Phase Transitions in a Constrained Two-Qubit Quantum Control Landscape
• Faster algorithms for the alignment of sparse correlated Erdös-Rényi random graphs
• Minority Takeover in Majority Dynamics: Searching for Rare Initializations via the History Passing Algorithm
• Landscape Complexity for the Empirical Risk of Generalized Linear Models: Discrimination between Structured Data
• Precise Error Rates for Computationally Efficient Testing
• Interacting Copies of Random Constraint Satisfaction Problems
• The maximum-average subtensor problem: equilibrium and out-of-equilibrium properties