Deep Boltzmann machines: rigorous results at arbitrary depth
arXiv:2004.04495 · doi:10.1007/s00023-021-01027-2
Abstract
A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a -layers spin glass model is studied allowing interactions only among consecutive layers. A lower bound for the pressure is found in terms of a convex combination of Sherrington-Kirkpatrick models and used to study the annealed and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously control the annealed region at arbitrary depth with the methods introduced by Heilmann and Lieb. The compression of this high noise region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the limiting quenched pressure of the model is obtained in a suitable region of the parameters and the replica symmetric pressure is proved to have a unique stationary point.
28 pages
References in corpus (3)
Cited by in corpus (10)
- Free energy in multi-species mixed -spin spherical models
- The solution of the deep Boltzmann machine on the Nishimori line
- The multi-species mean-field spin-glass on the Nishimori line
- Fluctuation results for Multi-species Sherrington-Kirkpatrick model in the replica symmetric regime
- Hopfield model with planted patterns: a teacher-student self-supervised learning model
- Statistical-mechanical study of deep Boltzmann machine given weight parameters after training by singular value decomposition
- On the free energy of vector spin glasses with non-convex interactions
- Convergence of the free energy for spherical spin glasses
- Pattern recognition in Deep Boltzmann machines
- Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey