Black holes and the loss landscape in machine learning
arXiv:2306.14817 · doi:10.1007/JHEP10(2023)107
Abstract
Understanding the loss landscape is an important problem in machine learning. One key feature of the loss function, common to many neural network architectures, is the presence of exponentially many low lying local minima. Physical systems with similar energy landscapes may provide useful insights. In this work, we point out that black holes naturally give rise to such landscapes, owing to the existence of black hole entropy. For definiteness, we consider 1/8 BPS black holes in string theory. These provide an infinite family of potential landscapes arising in the microscopic descriptions of corresponding black holes. The counting of minima amounts to black hole microstate counting. Moreover, the exact numbers of the minima for these landscapes are a priori known from dualities in string theory. Some of the minima are connected by paths of low loss values, resembling mode connectivity. We estimate the number of runs needed to find all the solutions. Initial explorations suggest that Stochastic Gradient Descent can find a significant fraction of the minima.
arXiv comment32 pages, 4 figures
References in corpus (45)
- Microscopic Origin of the Bekenstein-Hawking Entropy
- Machine learning and the physical sciences
- Theoretical perspective on the glass transition and amorphous materials
- Machine learning phases of matter
- Black Hole Entropy in M-Theory
- Quantum Quivers and Hall/Hole Halos
- The statistics of critical points of Gaussian fields on large-dimensional spaces
- Complexity of random smooth functions on the high-dimensional sphere
- Machine Learning in the String Landscape
- Supersymmetric Index from Black Hole Entropy
- Perspective: Energy Landscapes for Machine Learning
- Evolving neural networks with genetic algorithms to study the String Landscape
- Machine Learning for Condensed Matter Physics
- Machine Learning CICY Threefolds
- Counting Dyons in N=8 String Theory
- Scaling BPS Solutions and pure-Higgs States
- Neural Networks and Quantum Field Theory
- Deep Learning and Holographic QCD
- F^4 Terms in N=4 String Vacua
- AdS/CFT as a deep Boltzmann machine
- Deep Learning and AdS/QCD
- Deep learning in the heterotic orbifold landscape
- Comparing Dynamics: Deep Neural Networks versus Glassy Systems
- Getting CICY High
- Quiver Mutations, Seiberg Duality and Machine Learning
- Solving Conformal Field Theories with Artificial Intelligence
- Machine Learning Calabi-Yau Hypersurfaces
- Arithmetic of N=8 Black Holes
- Nonperturbative renormalization for the neural network-QFT correspondence
- Mean-field theory for a spin-glass model of neural networks: TAP free energy and paramagnetic to spin-glass transition
- How Do Black Holes Predict the Sign of the Fourier Coefficients of Siegel Modular Forms?
- Machine Learning Calabi-Yau Four-folds
- Is Deep Learning a Renormalization Group Flow?
- Deep learning black hole metrics from shear viscosity
- Machine learning for complete intersection Calabi-Yau manifolds: a methodological study
- Inception Neural Network for Complete Intersection Calabi-Yau 3-folds
- Machine Learning Lie Structures & Applications to Physics
- 6D (2,0) Bootstrap with soft-Actor-Critic
- Deep learning the holographic black hole with charge
- Fundamental problems in statistical physics XIV: Lecture on Machine Learning
- BPS State Counting in N=8 Supersymmetric String Theory for Pure D-brane Configurations
- Disentangling a Deep Learned Volume Formula
- Connecting Dualities and Machine Learning
- Multi-centered black holes, scaling solutions and pure-Higgs indices from localization
- Learning knot invariants across dimensions