Efficient training of energy-based models via spin-glass control

Abstract

We introduce a new family of energy-based probabilistic graphical models for efficient unsupervised learning. Its definition is motivated by the control of the spin-glass properties of the Ising model described by the weights of Boltzmann machines. We use it to learn the Bars and Stripes dataset of various sizes and the MNIST dataset, and show how they quickly achieve the performance offered by standard methods for unsupervised learning. Our results indicate that the standard initialization of Boltzmann machines with random weights equivalent to spin-glass models is an unnecessary bottleneck in the process of training. Furthermore, this new family allows for very easy access to low-energy configurations, which points to new, efficient training algorithms. The simplest variant of such algorithms approximates the negative phase of the log-likelihood gradient with no Markov chain Monte Carlo sampling costs at all, and with an accuracy sufficient to achieve good learning and generalization.

arXiv comment16 pages, 7+1 figures, RevTeX 4.1. Code is available at https://github.com/apozas/rapid/. v4: Updated to match published version

References in corpus (7)

• Simulated Tempering: A New Monte Carlo Scheme
• Quantum Boltzmann Machine
• Opportunities and challenges for quantum-assisted machine learning in near-term quantum computers
• From principal component to direct coupling analysis of coevolution in proteins: Low-eigenvalue modes are needed for structure prediction
• Spectral Dynamics of Learning Restricted Boltzmann Machines
• Selection of sequence motifs and generative Hopfield-Potts models for protein familiesilies
• Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networks

Cited by in corpus (3)

• Defence against adversarial attacks using classical and quantum-enhanced Boltzmann machines
• Privacy-preserving machine learning with tensor networks
• Unlearning regularization for Boltzmann Machines