The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning

Abstract

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

arXiv comment7+21 pages

References in corpus (83)

• Observation of Gravitational Waves from a Binary Black Hole Merger
• Quantum Machine Learning
• The density-matrix renormalization group in the age of matrix product states
• Quantum algorithm for solving linear systems of equations
• Area laws for the entanglement entropy - a review
• Machine learning and the physical sciences
• Barren plateaus in quantum neural network training landscapes
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Quantum machine learning in feature Hilbert spaces
• Machine learning phases of matter
• Quantum principal component analysis
• Predicting Many Properties of a Quantum System from Very Few Measurements
• Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits
• Symmetric Informationally Complete Quantum Measurements
• Matrix Product States and Projected Entangled Pair States: Concepts, Symmetries, and Theorems
• Many-body quantum state tomography with neural networks
• Exact and Approximate Unitary 2-Designs: Constructions and Applications
• Efficient variational quantum simulator incorporating active error minimisation
• Integration with respect to the Haar measure on unitary, orthogonal and symplectic group
• Discovering Phase Transitions with Unsupervised Learning
• Criticality, the area law, and the computational power of PEPS
• Quantum generative adversarial learning
• Tensor networks for complex quantum systems
• A rigorous and robust quantum speed-up in supervised machine learning
• Quantum autoencoders for efficient compression of quantum data
• Holographic duality from random tensor networks
• Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1
• Quantum-enhanced machine learning
• Quantum Entanglement in Neural Network States
• Why does deep and cheap learning work so well?
• An initialization strategy for addressing barren plateaus in parametrized quantum circuits
• Absence of Barren Plateaus in Quantum Convolutional Neural Networks
• Quantum Boltzmann Machine
• Random Quantum Circuits are Approximate 2-designs
• Towards Quantum Machine Learning with Tensor Networks
• Identifying topological order through unsupervised machine learning
• Quantum Loop Topography for Machine Learning
• Machine Learning Topological Invariants with Neural Networks
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• Unsupervised Generative Modeling Using Matrix Product States
• Equivalence of restricted Boltzmann machines and tensor network states
• Variational Fast Forwarding for Quantum Simulation Beyond the Coherence Time
• Effect of barren plateaus on gradient-free optimization
• Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
• Entanglement Transitions from Holographic Random Tensor Networks
• Quantum Entanglement in Deep Learning Architectures
• Mutual Information, Neural Networks and the Renormalization Group
• Trainability of Dissipative Perceptron-Based Quantum Neural Networks
• Quantum generative adversarial learning in a superconducting quantum circuit
• Unsupervised machine learning and band topology
• Machine learning meets quantum physics
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• Entanglement Devised Barren Plateau Mitigation
• Quantum Adversarial Machine Learning
• Barren plateaus preclude learning scramblers
• Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus
• On barren plateaus and cost function locality in variational quantum algorithms
• Variational Quantum Fidelity Estimation
• Application of machine learning algorithms to the study of noise artifacts in gravitational-wave data
• Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure
• Learning Relevant Features of Data with Multi-scale Tensor Networks
• Topological Quantum Compiling with Reinforcement Learning
• Variational Quantum Unsampling on a Quantum Photonic Processor
• Enhancing Generative Models via Quantum Correlations
• Analyzing the barren plateau phenomenon in training quantum neural networks with the ZX-calculus
• Machine Learning Bell Nonlocality in Quantum Many-body Systems
• Supervised Learning with Projected Entangled Pair States
• Typicality in random matrix product states
• Quantum Natural Language Processing on Near-Term Quantum Computers
• Precision-guaranteed quantum tomography
• Quantum computational webs
• Matrix Product Operators for Sequence to Sequence Learning
• Unsupervised Learning of Non-Hermitian Topological Phases
• Tensor Network Contractions for #SAT
• Statistical properties of random matrix product states
• Holographic coherent states from random tensor networks
• Generative machine learning with tensor networks: benchmarks on near-term quantum computers
• Guaranteed recovery of quantum processes from few measurements
• Emergent statistical mechanics from properties of disordered random matrix product states
• Matrix Product States, Random Matrix Theory and the Principle of Maximum Entropy
• Quantum Machine Learning Tensor Network States
• Mutual Information Scaling for Tensor Network Machine Learning
• Tangent-Space Gradient Optimization of Tensor Network for Machine Learning

Cited by in corpus (31)

• Barren Plateaus in Variational Quantum Computing
• Quantum Computing for High-Energy Physics: State of the Art and Challenges. Summary of the QC4HEP Working Group
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• A Lie Algebraic Theory of Barren Plateaus for Deep Parameterized Quantum Circuits
• Recent advances for quantum classifiers
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• Biology and medicine in the landscape of quantum advantages
• Barren plateaus in quantum tensor network optimization
• Does provable absence of barren plateaus imply classical simulability?
• Subtleties in the trainability of quantum machine learning models
• Analytic theory for the dynamics of wide quantum neural networks
• Constant-depth preparation of matrix product states with adaptive quantum circuits
• Tensor networks for quantum machine learning
• Absence of barren plateaus in finite local-depth circuits with long-range entanglement
• Trainability Enhancement of Parameterized Quantum Circuits via Reduced-Domain Parameter Initialization
• Quantum Capsule Networks
• Training variational quantum algorithms with random gate activation
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning
• Quantum Convolutional Neural Networks are Effectively Classically Simulable
• Isometric tensor network optimization for extensive Hamiltonians is free of barren plateaus
• Barren plateaus from learning scramblers with local cost functions
• Ground state search by local and sequential updates of neural network quantum states
• Magic of Random Matrix Product States
• Machine learning with tree tensor networks, CP rank constraints, and tensor dropout
• Absence of barren plateaus and scaling of gradients in the energy optimization of isometric tensor network states
• Typical Machine Learning Datasets as Low-Depth Quantum Circuits
• The Dual Role of Low-Weight Pauli Propagation: A Flawed Simulator but a Powerful Initializer for Variational Quantum Algorithms
• The informativeness of the gradient revisited
• Equivalence between exponential concentration in quantum machine learning kernels and barren plateaus in variational algorithms
• Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks
• Matrix Product State on a Quantum Computer