Advanced Mean Field Theory of Restricted Boltzmann Machine

Abstract

Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message passing based method that evaluates not only the partition function (free energy) but also its gradients without requiring statistical sampling. The results are compared with those obtained by the computationally expensive sampling based method.

5 pages, 4 figures, accepted by Phys Rev E (Rapid Communication)

Cited by in corpus (6)

• Mean-field message-passing equations in the Hopfield model and its generalizations
• Spectral Dynamics of Learning Restricted Boltzmann Machines
• Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
• Singular-Value-Decomposition Analysis of Associative Memory in a Neural Network
• Thermodynamics of the Ising model encoded in restricted Boltzmann machines
• Efficient Evaluation of the Partition Function of RBMs with Annealed Importance Sampling