Thermodynamics of the Ising model encoded in restricted Boltzmann machines
arXiv:2210.06203 · doi:10.3390/e24121701
Abstract
The restricted Boltzmann machine (RBM) is a two-layer energy-based model that uses its hidden-visible connections to learn the underlying distribution of visible units, whose interactions are often complicated by high-order correlations. Previous studies on the Ising model of small system sizes have shown that RBMs are able to accurately learn the Boltzmann distribution and reconstruct thermal quantities at temperatures away from the critical point . How the RBM encodes the Boltzmann distribution and captures the phase transition are, however, not well explained. In this work, we perform RBM learning of the and Ising model and carefully examine how the RBM extracts useful probabilistic and physical information from Ising configurations. We find several indicators derived from the weight matrix that could characterize the Ising phase transition. We verify that the hidden encoding of a visible state tends to have an equal number of positive and negative units, whose sequence is randomly assigned during training and can be inferred by analyzing the weight matrix. We also explore the physical meaning of visible energy and loss function (pseudo-likelihood) of the RBM and show that they could be harnessed to predict the critical point or estimate physical quantities such as entropy.
arXiv comment13 pages, 8 figures
References in corpus (38)
- Machine learning and the physical sciences
- Solving the Quantum Many-Body Problem with Artificial Neural Networks
- Machine learning phases of matter
- Discovering Phase Transitions with Unsupervised Learning
- Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders
- Discovering physical concepts with neural networks
- Statistical physics of inference: Thresholds and algorithms
- Why does deep and cheap learning work so well?
- Machine learning quantum phases of matter beyond the fermion sign problem
- Discovering Phases, Phase Transitions and Crossovers through Unsupervised Machine Learning: A critical examination
- Inverse statistical problems: from the inverse Ising problem to data science
- Learning Thermodynamics with Boltzmann Machines
- Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
- Equivalence of restricted Boltzmann machines and tensor network states
- The Principles of Deep Learning Theory
- Solving Statistical Mechanics Using Variational Autoregressive Networks
- Detection of phase transition via convolutional neural network
- Inverse Ising inference using all the data
- Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
- The jamming transition as a paradigm to understand the loss landscape of deep neural networks
- Asymptotically unbiased estimation of physical observables with neural samplers
- Perspective: Energy Landscapes for Machine Learning
- Machine Learning for Condensed Matter Physics
- Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
- Deep Learning the Ising Model Near Criticality
- Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning
- Thermodynamics of Restricted Boltzmann Machines and related learning dynamics
- Thermodynamics and Feature Extraction by Machine Learning
- Comparing Dynamics: Deep Neural Networks versus Glassy Systems
- Learning the Ising Model with Generative Neural Networks
- Machine learning determination of dynamical parameters: The Ising model case
- Advanced Mean Field Theory of Restricted Boltzmann Machine
- Unsupervised and supervised learning of interacting topological phases from single-particle correlation functions
- Is Deep Learning a Renormalization Group Flow?
- Phase transition encoded in neural network
- Nonequilibrium Thermodynamics of Restricted Boltzmann Machines
- Identifying Product Order with Restricted Boltzmann Machines
- Entropy, Free Energy, and Work of Restricted Boltzmann Machines