Boltzmann machines as two-dimensional tensor networks

Abstract

Restricted Boltzmann machines (RBM) and deep Boltzmann machines (DBM) are important models in machine learning, and recently found numerous applications in quantum many-body physics. We show that there are fundamental connections between them and tensor networks. In particular, we demonstrate that any RBM and DBM can be exactly represented as a two-dimensional tensor network. This representation gives an understanding of the expressive power of RBM and DBM using entanglement structures of the tensor networks, also provides an efficient tensor network contraction algorithm for the computing partition function of RBM and DBM. Using numerical experiments, we demonstrate that the proposed algorithm is much more accurate than the state-of-the-art machine learning methods in estimating the partition function of restricted Boltzmann machines and deep Boltzmann machines, and have potential applications in training deep Boltzmann machines for general machine learning tasks.

arXiv comment12 pages, 11 figures

References in corpus (19)

• The density-matrix renormalization group in the age of matrix product states
• Machine learning and the physical sciences
• Efficient classical simulation of slightly entangled quantum computations
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Many-body quantum state tomography with neural networks
• Tensor renormalization group approach to 2D classical lattice models
• Quantum Entanglement in Neural Network States
• Coarse-graining renormalization by higher-order singular value decomposition
• Restricted-Boltzmann-Machine Learning for Solving Strongly Correlated Quantum Systems
• On Tensors, Sparsity, and Nonnegative Factorizations
• Accelerate Monte Carlo Simulations with Restricted Boltzmann Machines
• Equivalence of restricted Boltzmann machines and tensor network states
• Neural-Network Quantum States, String-Bond States, and Chiral Topological States
• Loop optimization for tensor network renormalization
• Constructing exact representations of quantum many-body systems with deep neural networks
• Efficient Representation of Topologically Ordered States with Restricted Boltzmann Machines
• Contracting Arbitrary Tensor Networks: General Approximate Algorithm and Applications in Graphical Models and Quantum Circuit Simulations
• Advanced Mean Field Theory of Restricted Boltzmann Machine
• Training Restricted Boltzmann Machines via the Thouless-Anderson-Palmer Free Energy

Cited by in corpus (9)

• Neural network representation of tensor network and chiral states
• Tensor networks for quantum machine learning
• Tensor networks for unsupervised machine learning
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Tensor networks for quantum computing
• A Non-stochastic Optimization Algorithm for Neural-network Quantum States
• Machine Learning Wavefunction
• Neural Quantum State Study of Fracton Models
• Seeding neural network quantum states with tensor network states