Machine learning with tree tensor networks, CP rank constraints, and tensor dropout

Abstract

Tensor networks developed in the context of condensed matter physics try to approximate order-$N$ tensors with a reduced number of degrees of freedom that is only polynomial in $N$ and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks [arXiv:2205.15296]. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio $b=4$ reaches a test set accuracy of 90.3\% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.

7 pages, 8 figures; published version

References in corpus (41)

• Adam: A Method for Stochastic Optimization
• XGBoost: A Scalable Tree Boosting System
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• The density-matrix renormalization group in the age of matrix product states
• A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
• On the Convergence of Adam and Beyond
• Entanglement renormalization
• A class of quantum many-body states that can be efficiently simulated
• Criticality, the area law, and the computational power of PEPS
• Classical simulation of quantum many-body systems with a tree tensor network
• Renormalization algorithms for Quantum-Many Body Systems in two and higher dimensions
• Simulation of two dimensional quantum systems on an infinite lattice revisited: corner transfer matrix for tensor contraction
• Hierarchical quantum classifiers
• Towards Quantum Machine Learning with Tensor Networks
• Unsupervised Generative Modeling Using Matrix Product States
• Simulating Strongly Correlated Quantum Systems with Tree Tensor Networks
• Efficient Tree Tensor Network States (TTNS) for Quantum Chemistry: Generalizations of the Density Matrix Renormalization Group Algorithm
• Tree Tensor Networks for Generative Modeling
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• Learning Feynman Diagrams with Tensor Trains
• TensorNetwork for Machine Learning
• Supervised Learning with Projected Entangled Pair States
• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• Generative Tensor Network Classification Model for Supervised Machine Learning
• Tensor networks for unsupervised machine learning
• Entanglement-guided architectures of machine learning by quantum tensor network
• Hybrid quantum-classical classifier based on tensor network and variational quantum circuit
• Mutual Information Scaling for Tensor Network Machine Learning
• Isometric tensor network optimization for extensive Hamiltonians is free of barren plateaus
• Quantum-classical eigensolver using multiscale entanglement renormalization
• Unsupervised Recognition of Informative Features via Tensor Network Machine Learning and Quantum Entanglement Variations
• On the closedness and geometry of tensor network state sets
• Quantum-inspired event reconstruction with Tensor Networks: Matrix Product States
• Quantum Tensor Network in Machine Learning: An Application to Tiny Object Classification
• Generative modeling with projected entangled-pair states
• Interaction Decompositions for Tensor Network Regression
• Tensor network to learn the wavefunction of data
• Generalization and Overfitting in Matrix Product State Machine Learning Architectures
• Quantum state tomography with tensor train cross approximation
• Tensor Network States with Low-Rank Tensors

Cited by in corpus (4)

• Universal scaling laws in quantum-probabilistic machine learning by tensor network towards interpreting representation and generalization powers
• Scaling of contraction costs for entanglement renormalization algorithms including tensor Trotterization and variational Monte Carlo
• Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems
• Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks