Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning

Abstract

Tensor-network techniques have enjoyed outstanding success in physics, and have recently attracted attention in machine learning, both as a tool for the formulation of new learning algorithms and for enhancing the mathematical understanding of existing methods. Inspired by these developments, and the natural correspondence between tensor networks and probabilistic graphical models, we provide a rigorous analysis of the expressive power of various tensor-network factorizations of discrete multivariate probability distributions. These factorizations include non-negative tensor-trains/MPS, which are in correspondence with hidden Markov models, and Born machines, which are naturally related to local quantum circuits. When used to model probability distributions, they exhibit tractable likelihoods and admit efficient learning algorithms. Interestingly, we prove that there exist probability distributions for which there are unbounded separations between the resource requirements of some of these tensor-network factorizations. Particularly surprising is the fact that using complex instead of real tensors can lead to an arbitrarily large reduction in the number of parameters of the network. Additionally, we introduce locally purified states (LPS), a new factorization inspired by techniques for the simulation of quantum systems, with provably better expressive power than all other representations considered. The ramifications of this result are explored through numerical experiments. Our findings imply that LPS should be considered over hidden Markov models, and furthermore provide guidelines for the design of local quantum circuits for probabilistic modeling.

arXiv comment14 pages + 14 pages supplementary material, extended version, code available at http://github.com/glivan/tensor_networks_for_probabilistic_modeling

References in corpus (1)

• Purifications of multipartite states: limitations and constructive methods

Cited by in corpus (16)

• Quantum Quasi-Monte Carlo Technique for Many-Body Perturbative Expansions
• Quantum Transfer Learning for Real-World, Small, and High-Dimensional Datasets
• Tree tensor network classifiers for machine learning: from quantum-inspired to quantum-assisted
• Generative machine learning with tensor networks: benchmarks on near-term quantum computers
• Potential and limitations of random Fourier features for dequantizing quantum machine learning
• Tensor network approaches for learning non-linear dynamical laws
• Mixed states in one spatial dimension: decompositions and correspondence with nonnegative matrices
• Tensor decompositions on simplicial complexes with invariance
• A Multi-Scale Tensor Network Architecture for Classification and Regression
• Tensor network to learn the wavefunction of data
• Tensor Networks for Simulating Quantum Circuits on FPGAs
• The resource theory of tensor networks
• A quantum inspired approach to learning dynamical laws from data -- block-sparsity and gauge-mediated weight sharing
• Border Ranks of Positive and Invariant Tensor Decompositions: Applications to Correlations
• Approximate tensor decompositions: disappearance of many separations
• SchrödingeRNN: Generative Modeling of Raw Audio as a Continuously Observed Quantum State