Mutual Information Scaling for Tensor Network Machine Learning

Abstract

Tensor networks have emerged as promising tools for machine learning, inspired by their widespread use as variational ansatze in quantum many-body physics. It is well known that the success of a given tensor network ansatz depends in part on how well it can reproduce the underlying entanglement structure of the target state, with different network designs favoring different scaling patterns. We demonstrate here how a related correlation analysis can be applied to tensor network machine learning, and explore whether classical data possess correlation scaling patterns similar to those found in quantum states which might indicate the best network to use for a given dataset. We utilize mutual information as measure of correlations in classical data, and show that it can serve as a lower-bound on the entanglement needed for a probabilistic tensor network classifier. We then develop a logistic regression algorithm to estimate the mutual information between bipartitions of data features, and verify its accuracy on a set of Gaussian distributions designed to mimic different correlation patterns. Using this algorithm, we characterize the scaling patterns in the MNIST and Tiny Images datasets, and find clear evidence of boundary-law scaling in the latter. This quantum-inspired classical analysis offers insight into the design of tensor networks which are best suited for specific learning tasks.

arXiv comment29 pages, 10 figures

References in corpus (34)

• The density-matrix renormalization group in the age of matrix product states
• Estimating Mutual Information
• Area laws for the entanglement entropy - a review
• Entanglement in quantum critical phenomena
• Efficient classical simulation of slightly entangled quantum computations
• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• Matrix Product States, Projected Entangled Pair States, and variational renormalization group methods for quantum spin systems
• Average Entropy of a Subsystem
• Quantum Convolutional Neural Networks
• An Area Law for One Dimensional Quantum Systems
• A class of quantum many-body states that can be efficiently simulated
• Classical simulation of quantum many-body systems with a tree tensor network
• MINE: Mutual Information Neural Estimation
• Hand-waving and Interpretive Dance: An Introductory Course on Tensor Networks
• Towards Quantum Machine Learning with Tensor Networks
• Tensor network states and geometry
• An entanglement-area law for general bosonic harmonic lattice systems
• On Variational Bounds of Mutual Information
• On the Expressive Power of Deep Learning: A Tensor Analysis
• Correlations in local measurements on a quantum state, and complementarity as an explanation of nonclassicality
• Tensor Networks in a Nutshell
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Quantum Inference on Bayesian Networks
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• Supervised Learning with Quantum-Inspired Tensor Networks
• Efficient simulation of infinite tree tensor network states on the Bethe lattice
• Supervised Learning with Projected Entangled Pair States
• Entanglement and tensor network states
• Lectures on Quantum Tensor Networks
• An introduction to entanglement measures
• Tighter Variational Representations of f-Divergences via Restriction to Probability Measures
• Entanglement and Tensor Networks for Supervised Image Classification
• A Multi-Scale Tensor Network Architecture for Classification and Regression

Cited by in corpus (11)

• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• Tensor networks for quantum machine learning
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Entanglement bipartitioning and tree tensor networks
• Decohering Tensor Network Quantum Machine Learning Models
• Machine learning with tree tensor networks, CP rank constraints, and tensor dropout
• Tensor network to learn the wavefunction of data
• Tensor networks and efficient descriptions of classical data
• Tensor tree learns hidden relational structures in data to construct generative models
• Deep tensor networks with matrix product operators
• Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers