Black holes and the loss landscape in machine learning
arXiv:2306.14817 · doi:10.1007/JHEP10(2023)107
Abstract
Understanding the loss landscape is an important problem in machine learning. One key feature of the loss function, common to many neural network architectures, is the presence of exponentially many low lying local minima. Physical systems with similar energy landscapes may provide useful insights. In this work, we point out that black holes naturally give rise to such landscapes, owing to the existence of black hole entropy. For definiteness, we consider 1/8 BPS black holes in string theory. These provide an infinite family of potential landscapes arising in the microscopic descriptions of corresponding black holes. The counting of minima amounts to black hole microstate counting. Moreover, the exact numbers of the minima for these landscapes are a priori known from dualities in string theory. Some of the minima are connected by paths of low loss values, resembling mode connectivity. We estimate the number of runs needed to find all the solutions. Initial explorations suggest that Stochastic Gradient Descent can find a significant fraction of the minima.
32 pages, 4 figures
References in corpus (12)
- Theoretical perspective on the glass transition and amorphous materials
- Machine Learning in the String Landscape
- Supersymmetric Index from Black Hole Entropy
- Evolving neural networks with genetic algorithms to study the String Landscape
- AdS/CFT as a deep Boltzmann machine
- Deep Learning and AdS/QCD
- Getting CICY High
- Machine Learning Calabi-Yau Hypersurfaces
- How Do Black Holes Predict the Sign of the Fourier Coefficients of Siegel Modular Forms?
- Machine Learning Lie Structures & Applications to Physics
- 6D (2,0) Bootstrap with soft-Actor-Critic
- BPS State Counting in N=8 Supersymmetric String Theory for Pure D-brane Configurations