Pattern recognition in Deep Boltzmann machines

Abstract

We consider a multi-layer Sherrington-Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines and we solve for its quenched free energy, in the thermodynamic limit and allowing for a first step of replica symmetry breaking. This result is accomplished rigorously exploiting interpolating techniques and recovering the expression already known for the replica-symmetry case. Further, we drop the restriction constraint by introducing intra-layer connections among spins and we show that the resulting system can be mapped into a modular Hopfield network, which is also addressed rigorously via interpolating techniques up to the first step of replica symmetry breaking.

arXiv comment24 pages, 2 figures

References in corpus (7)

• Griffiths phases and the stretching of criticality in brain networks
• Phase transitions in Restricted Boltzmann Machines with generic priors
• From Dyson to Hopfield: Processing on hierarchical networks
• Universality in bipartite mean field spin glasses
• Annealing and replica-symmetry in Deep Boltzmann Machines
• Deep Boltzmann machines: rigorous results at arbitrary depth
• Replica symmetry breaking in multi-species Sherrington-Kirkpatrick model

Cited by in corpus (1)

• Free energy in multi-species mixed $p$-spin spherical models