The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Abstract

Generative Autoregressive Neural Networks (ARNNs) have recently demonstrated exceptional results in image and language generation tasks, contributing to the growing popularity of generative models in both scientific and commercial applications. This work presents an exact mapping of the Boltzmann distribution of binary pairwise interacting systems into autoregressive form. The resulting ARNN architecture has weights and biases of its first layer corresponding to the Hamiltonian's couplings and external fields, featuring widely used structures such as the residual connections and a recurrent architecture with clear physical meanings. Moreover, its architecture's explicit formulation enables the use of statistical physics techniques to derive new ARNNs for specific systems. As examples, new effective ARNN architectures are derived from two well-known mean-field systems, the Curie-Weiss and Sherrington-Kirkpatrick models, showing superior performance in approximating the Boltzmann distributions of the corresponding physics model compared to other commonly used architectures. The connection established between the physics of the system and the neural network architecture provides a means to derive new architectures for different interacting systems and interpret existing ones from a physical perspective.

20 pages, 10 figure plus the Supplementary Information

References in corpus (4)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning phase transitions by confusion
• Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems
• A Bayesian generative neural network framework for epidemic inference problems

Cited by in corpus (7)

• Sparse Autoregressive Neural Networks for Classical Spin Systems
• Rényi entanglement entropy of spin chain with Generative Neural Networks
• Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models
• Efficient Optimization of Variational Autoregressive Networks with Natural Gradient
• Nearest-Neighbours Neural Network architecture for efficient sampling of statistical physics models
• Neural-network-assisted Monte Carlo sampling trained by Quantum Approximate Optimization Algorithm
• Hierarchical autoregressive neural networks in three-dimensional statistical system