Deep Boltzmann machines: rigorous results at arbitrary depth

Abstract

A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a $K$-layers spin glass model is studied allowing interactions only among consecutive layers. A lower bound for the pressure is found in terms of a convex combination of $K$ Sherrington-Kirkpatrick models and used to study the annealed and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously control the annealed region at arbitrary depth $K$ with the methods introduced by Heilmann and Lieb. The compression of this high noise region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the limiting quenched pressure of the model is obtained in a suitable region of the parameters and the replica symmetric pressure is proved to have a unique stationary point.

arXiv comment28 pages

References in corpus (12)

• Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model
• The Thermodynamic Limit in Mean Field Spin Glass Models
• The Parisi formula has a unique minimizer
• The free energy in a multi-species Sherrington-Kirkpatrick model
• Equilibrium statistical mechanics of bipartite spin systems
• Quadratic replica coupling in the Sherrington-Kirkpatrick mean field spin glass model
• About the Almeida-Thouless transition line in the Sherrington-Kirkpatrick mean field spin glass mode
• Free energy and complexity of spherical bipartite models
• Some Properties of the Phase Diagram for Mixed $p$-Spin Glasses
• Scaling Limits for Multispecies Statistical Mechanics Mean-Field Models
• Annealing and replica-symmetry in Deep Boltzmann Machines
• Replica symmetry breaking in multi-species Sherrington-Kirkpatrick model

Cited by in corpus (10)

• Free energy in multi-species mixed $p$-spin spherical models
• The solution of the deep Boltzmann machine on the Nishimori line
• The multi-species mean-field spin-glass on the Nishimori line
• Fluctuation results for Multi-species Sherrington-Kirkpatrick model in the replica symmetric regime
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Statistical-mechanical study of deep Boltzmann machine given weight parameters after training by singular value decomposition
• On the free energy of vector spin glasses with non-convex interactions
• Convergence of the free energy for spherical spin glasses
• Pattern recognition in Deep Boltzmann machines
• Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey