The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning

Abstract

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

7+21 pages

References in corpus (9)

• The density-matrix renormalization group in the age of matrix product states
• Quantum algorithm for solving linear systems of equations
• Criticality, the area law, and the computational power of PEPS
• Quantum-enhanced machine learning
• Fast Automated Analysis of Strong Gravitational Lenses with Convolutional Neural Networks
• Machine learning meets quantum physics
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• Emergent statistical mechanics from properties of disordered random matrix product states
• Mutual Information Scaling for Tensor Network Machine Learning

Cited by in corpus (31)

• Barren Plateaus in Variational Quantum Computing
• Quantum Computing for High-Energy Physics: State of the Art and Challenges. Summary of the QC4HEP Working Group
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• A Lie Algebraic Theory of Barren Plateaus for Deep Parameterized Quantum Circuits
• Recent advances for quantum classifiers
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• Biology and medicine in the landscape of quantum advantages
• Barren plateaus in quantum tensor network optimization
• Does provable absence of barren plateaus imply classical simulability?
• Subtleties in the trainability of quantum machine learning models
• Analytic theory for the dynamics of wide quantum neural networks
• Constant-depth preparation of matrix product states with adaptive quantum circuits
• Tensor networks for quantum machine learning
• Absence of barren plateaus in finite local-depth circuits with long-range entanglement
• Trainability Enhancement of Parameterized Quantum Circuits via Reduced-Domain Parameter Initialization
• Quantum Capsule Networks
• Training variational quantum algorithms with random gate activation
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning
• Quantum Convolutional Neural Networks are Effectively Classically Simulable
• Isometric tensor network optimization for extensive Hamiltonians is free of barren plateaus
• Barren plateaus from learning scramblers with local cost functions
• Ground state search by local and sequential updates of neural network quantum states
• Magic of Random Matrix Product States
• Machine learning with tree tensor networks, CP rank constraints, and tensor dropout
• Absence of barren plateaus and scaling of gradients in the energy optimization of isometric tensor network states
• Typical Machine Learning Datasets as Low-Depth Quantum Circuits
• The Dual Role of Low-Weight Pauli Propagation: A Flawed Simulator but a Powerful Initializer for Variational Quantum Algorithms
• The informativeness of the gradient revisited
• Equivalence between exponential concentration in quantum machine learning kernels and barren plateaus in variational algorithms
• Trainable Quantum Neural Network for Multiclass Image Classification with the Power of Pre-trained Tree Tensor Networks
• Matrix Product State on a Quantum Computer