Tree Tensor Networks for Generative Modeling

Abstract

Matrix product states (MPS), a tensor network designed for one-dimensional quantum systems, has been recently proposed for generative modeling of natural data (such as images) in terms of `Born machine'. However, the exponential decay of correlation in MPS restricts its representation power heavily for modeling complex data such as natural images. In this work, we push forward the effort of applying tensor networks to machine learning by employing the Tree Tensor Network (TTN) which exhibits balanced performance in expressibility and efficient training and sampling. We design the tree tensor network to utilize the 2-dimensional prior of the natural images and develop sweeping learning and sampling algorithms which can be efficiently implemented utilizing Graphical Processing Units (GPU). We apply our model to random binary patterns and the binary MNIST datasets of handwritten digits. We show that TTN is superior to MPS for generative modeling in keeping correlation of pixels in natural images, as well as giving better log-likelihood scores in standard datasets of handwritten digits. We also compare its performance with state-of-the-art generative models such as the Variational AutoEncoders, Restricted Boltzmann machines, and PixelCNN. Finally, we discuss the future development of Tensor Network States in machine learning problems.

References in corpus (25)

• Auto-Encoding Variational Bayes
• The density-matrix renormalization group in the age of matrix product states
• Area laws for the entanglement entropy - a review
• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• Entanglement renormalization
• Tensor renormalization group approach to 2D classical lattice models
• Classical simulation of quantum many-body systems with a tree tensor network
• Renormalization algorithms for Quantum-Many Body Systems in two and higher dimensions
• Quantum Entanglement in Neural Network States
• Why does deep and cheap learning work so well?
• Efficient Representation of Quantum Many-body States with Deep Neural Networks
• Differentiable Learning of Quantum Circuit Born Machine
• Simulation of two-dimensional quantum systems using a tree tensor network that exploits the entropic area law
• Unsupervised Generative Modeling Using Matrix Product States
• Equivalence of restricted Boltzmann machines and tensor network states
• Importance Weighted Autoencoders
• Shannon and entanglement entropies of one- and two-dimensional critical wave functions
• Generic Construction of Efficient Matrix Product Operators
• Advances on Tensor Network Theory: Symmetries, Fermions, Entanglement, and Holography
• Perfect Sampling with Unitary Tensor Networks
• Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
• Supervised Learning with Quantum-Inspired Tensor Networks
• An efficient quantum algorithm for generative machine learning
• Variational Monte Carlo with the Multi-Scale Entanglement Renormalization Ansatz
• Shortcut Matrix Product States and its applications

Cited by in corpus (69)

• Machine learning and the physical sciences
• Differentiable Programming Tensor Networks
• Lecture Notes of Tensor Network Contractions
• Yao.jl: Extensible, Efficient Framework for Quantum Algorithm Design
• Variational Quantum Reinforcement Learning via Evolutionary Optimization
• An end-to-end trainable hybrid classical-quantum classifier
• Contracting Arbitrary Tensor Networks: General Approximate Algorithm and Applications in Graphical Models and Quantum Circuit Simulations
• Supervised Learning with Projected Entangled Pair States
• Generative Tensor Network Classification Model for Supervised Machine Learning
• Trainability barriers and opportunities in quantum generative modeling
• Tensor networks for quantum machine learning
• Accurate simulation for finite projected entangled pair states in two dimensions
• Density Matrix Renormalization Group with Tensor Processing Units
• Direct sampling of projected entangled-pair states
• Tensor networks for unsupervised machine learning
• From Tensor Network Quantum States to Tensorial Recurrent Neural Networks
• Automatic structural optimization of tree tensor networks
• Tree tensor network classifiers for machine learning: from quantum-inspired to quantum-assisted
• Generative machine learning with tensor networks: benchmarks on near-term quantum computers
• Probabilistic Modeling with Matrix Product States
• Entanglement-guided architectures of machine learning by quantum tensor network
• Modeling Sequences with Quantum States: A Look Under the Hood
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Simulating quantum circuits using tree tensor networks
• Experimental realization of a quantum image classifier via tensor-network-based machine learning
• Quantum Compressed Sensing with Unsupervised Tensor-Network Machine Learning
• Tensor networks for quantum computing
• Anomaly Detection with Tensor Networks
• Tangent-Space Gradient Optimization of Tensor Network for Machine Learning
• Unsupervised Recognition of Informative Features via Tensor Network Machine Learning and Quantum Entanglement Variations
• Machine learning with tree tensor networks, CP rank constraints, and tensor dropout
• Stack operation of tensor networks
• Residual Matrix Product State for Machine Learning
• Solving Quantum Statistical Mechanics with Variational Autoregressive Networks and Quantum Circuits
• Tensor network to learn the wavefunction of data
• Network-Initialized Monte Carlo Based on Generative Neural Networks
• Tensor Networks for Probabilistic Sequence Modeling
• Probabilistic Inference in the Era of Tensor Networks and Differential Programming
• Correlated states in super-moiré materials with a kernel polynomial quantics tensor cross interpolation algorithm
• QSlack: A slack-variable approach for variational quantum semi-definite programming
• Non-parametric Semi-Supervised Learning in Many-body Hilbert Space with Rescaled Logarithmic Fidelity
• Tensor tree learns hidden relational structures in data to construct generative models
• A Matrix Product State Model for Simultaneous Classification and Generation
• Exploring explicit coarse-grained structure in artificial neural networks
• Tensor networks and efficient descriptions of classical data
• Bayesian Tensor Network with Polynomial Complexity for Probabilistic Machine Learning
• Deep tensor networks with matrix product operators
• Survey on Computational Applications of Tensor Network Simulations
• Efficient Optimization of Variational Autoregressive Networks with Natural Gradient
• TensorKrowch: Smooth integration of tensor networks in machine learning
• Tensor-Networks-based Learning of Probabilistic Cellular Automata Dynamics
• Quantum tensor network algorithms for evaluation of spectral functions on quantum computers
• Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers
• Improving accuracy of tree-tensor network approach by optimization of network structure
• Subtractive Mixture Models via Squaring: Representation and Learning
• Tensor Network Estimation of Distribution Algorithms
• Multi-layered tensor networks for image classification
• Tensor Network for Supervised Learning at Finite Temperature
• Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems
• Explaining Anomalies with Tensor Networks
• Time series generation for option pricing on quantum computers using tensor network
• TTNOpt: Tree tensor network package for high-rank tensor compression
• Group-invariant tensor train networks for supervised learning
• Hybrid between biologically and quantum-inspired many-body states
• Tree tensor network impurity solver based on Cayley-tree mapping
• Using matrix-product states for time-series machine learning
• Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks
• SchrödingeRNN: Generative Modeling of Raw Audio as a Continuously Observed Quantum State
• Tensorization of neural networks for improved privacy and interpretability