Deep Learning the Ising Model Near Criticality

Abstract

It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural question to ask is whether deep neural networks hold any advantage over shallow networks in representing such data. We investigate this question by using unsupervised, generative graphical models to learn the probability distribution of a two-dimensional Ising system. Deep Boltzmann machines, deep belief networks, and deep restricted Boltzmann networks are trained on thermal spin configurations from this system, and compared to the shallow architecture of the restricted Boltzmann machine. We benchmark the models, focussing on the accuracy of generating energetic observables near the phase transition, where these quantities are most difficult to approximate. Interestingly, after training the generative networks, we observe that the accuracy essentially depends only on the number of neurons in the first hidden layer of the network, and not on other model details such as network depth or model type. This is evidence that shallow networks are more efficient than deep networks at representing physical probability distributions associated with Ising systems near criticality.

16 pages, 8 figures, 1 table

References in corpus (4)

• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• Equivalence of restricted Boltzmann machines and tensor network states
• Mutual Information, Neural Networks and the Renormalization Group
• An exact mapping between the Variational Renormalization Group and Deep Learning

Cited by in corpus (30)

• Machine learning and the physical sciences
• Identifying Quantum Phase Transitions with Adversarial Neural Networks
• Asymptotically unbiased estimation of physical observables with neural samplers
• Machine learning technique to find quantum many-body ground states of bosons on a lattice
• Parameter diagnostics of phases and phase transition learning by neural networks
• Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow
• Smallest Neural Network to Learn the Ising Criticality
• Efficient Modelling of Trivializing Maps for Lattice ${\varphi }^4$ Theory Using Normalizing Flows: A First Look at Scalability
• Machine Learning as a universal tool for quantitative investigations of phase transition
• Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines
• Is Deep Learning a Renormalization Group Flow?
• A perspective on machine learning and data science for strongly correlated electron problems
• Generative models for sampling and phase transition indication in spin systems
• Identifying Product Order with Restricted Boltzmann Machines
• Bayesian Optimization of Bose-Einstein Condensates
• Machine Learning of Nonequilibrium Phase Transition in an Ising Model on Square Lattice
• Machine learning algorithms based on generalized Gibbs ensembles
• The Ising Model with Hybrid Monte Carlo
• Variational Autoencoder Analysis of Ising Model Statistical Distributions and Phase Transitions
• Short sighted deep learning
• Conservation Laws and Spin System Modeling through Principal Component Analysis
• Thermodynamics of the Ising model encoded in restricted Boltzmann machines
• Minimalist Neural Networks training for phase classification in diluted-Ising models
• A sampling-guided unsupervised learning method to capture percolation in complex networks
• Reconstruction of Pairwise Interactions using Energy-Based Models
• Predictive Learning on Hidden Tree-Structured Ising Models
• Quantum Adiabatic Evolution for Global Optimization in Big Data
• Phase probabilities in first-order transitions using machine learning
• Self-regularizing restricted Boltzmann machines
• Learning disentangled representation for classical models