Tensor networks and efficient descriptions of classical data
arXiv:2103.06872 · doi:10.1103/PhysRevA.111.032409
Abstract
We investigate the potential of tensor network based machine learning methods to scale to large image and text data sets. For that, we study how the mutual information between a subregion and its complement scales with the subsystem size , similarly to how it is done in quantum many-body physics. We find that for text, the mutual information scales as a power law with a close to volume law exponent, indicating that text cannot be efficiently described by 1D tensor networks. For images, the scaling is close to an area law, hinting at 2D tensor networks such as PEPS could have an adequate expressibility. For the numerical analysis, we introduce a mutual information estimator based on autoregressive networks, and we also use convolutional neural networks in a neural estimator method.
21 pages, 6 figures; improvements and added a new model
References in corpus (48)
- The density-matrix renormalization group in the age of matrix product states
- Estimating Mutual Information
- Entanglement Entropy and Quantum Field Theory
- Area laws for the entanglement entropy - a review
- Entanglement in quantum critical phenomena
- Machine learning and the physical sciences
- Entanglement entropy and conformal field theory
- An Area Law for One Dimensional Quantum Systems
- A class of quantum many-body states that can be efficiently simulated
- Matrix Product States and Projected Entangled Pair States: Concepts, Symmetries, and Theorems
- Matrix product states represent ground states faithfully
- Area laws in quantum systems: mutual information and correlations
- Entanglement entropy in free quantum field theory
- Classical simulation of quantum many-body systems with a tree tensor network
- Entropy scaling and simulability by Matrix Product States
- Why does deep and cheap learning work so well?
- Efficient Representation of Quantum Many-body States with Deep Neural Networks
- The computational complexity of PEPS
- Towards Quantum Machine Learning with Tensor Networks
- Unsupervised Generative Modeling Using Matrix Product States
- Variational study of hard-core bosons in a 2-D optical lattice using Projected Entangled Pair States (PEPS)
- Solving Gapped Hamiltonians Locally
- Variational optimization with infinite projected entangled-pair states
- Stripes in the two-dimensional t-J model with infinite projected entangled-pair states
- Neural-Network Quantum States, String-Bond States, and Chiral Topological States
- Quantum Entanglement in Deep Learning Architectures
- Mutual Information, Neural Networks and the Renormalization Group
- Approximating Gibbs states of local Hamiltonians efficiently with PEPS
- Unifying Projected Entangled Pair States contractions
- Tree Tensor Networks for Generative Modeling
- On entropy growth and the hardness of simulating time evolution
- Three-sublattice order in the SU(3) Heisenberg model on the square and triangular lattice
- Finite Correlation Length Scaling in Lorentz-Invariant Gapless iPEPS Wave Functions
- Learning Relevant Features of Data with Multi-scale Tensor Networks
- Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
- Entanglement rates and area laws
- Period 4 stripe in the extended two-dimensional Hubbard model
- Enhancing Generative Models via Quantum Correlations
- Supervised Learning with Projected Entangled Pair States
- Critical properties of homogeneous binary trees
- Mutual information in classical spin models
- Quantum Approximate Markov Chains are Thermal
- Simulation of three-dimensional quantum systems with projected entangled-pair states
- Tensor networks for unsupervised machine learning
- Compressing deep neural networks by matrix product operators
- A scaling hypothesis for projected entangled-pair states
- Mutual Information Scaling for Tensor Network Machine Learning
- Language Design as Information Renormalization