Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus

Abstract

Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speedup applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speedup is the Barren Plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size $n$ for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with $n$. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training quantum neural networks (outside of the context of BPs).

5+1 pages, 1 figure, updated to published version

References in corpus (18)

• A Quantum Approximate Optimization Algorithm
• Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits
• The power of quantum neural networks
• Noise-Induced Barren Plateaus in Variational Quantum Algorithms
• The quest for a Quantum Neural Network
• Connecting ansatz expressibility to gradient magnitudes and barren plateaus
• Layerwise learning for quantum neural networks
• Solving nonlinear differential equations with differentiable quantum circuits
• Trainability of Dissipative Perceptron-Based Quantum Neural Networks
• Estimating the gradient and higher-order derivatives on quantum hardware
• Large gradients via correlation in random parameterized quantum circuits
• Variational Quantum State Eigensolver
• Barren plateaus preclude learning scramblers
• Quantum Assisted Simulator
• Learning to learn with quantum neural networks via classical neural networks
• Characterizing the loss landscape of variational quantum circuits
• Iterative Quantum Assisted Eigensolver
• MoG-VQE: Multiobjective genetic variational quantum eigensolver

Cited by in corpus (83)

• Variational Quantum Algorithms
• The Variational Quantum Eigensolver: a review of methods and best practices
• The power of quantum neural networks
• Noise-Induced Barren Plateaus in Variational Quantum Algorithms
• Challenges and Opportunities in Quantum Machine Learning
• Generalization in quantum machine learning from few training data
• Training variational quantum algorithms is NP-hard
• Absence of Barren Plateaus in Quantum Convolutional Neural Networks
• Effect of barren plateaus on gradient-free optimization
• Beyond Barren Plateaus: Quantum Variational Algorithms Are Swamped With Traps
• Diagnosing Barren Plateaus with Tools from Quantum Optimal Control
• Barren Plateaus in Variational Quantum Computing
• Quantum Computing for High-Energy Physics: State of the Art and Challenges. Summary of the QC4HEP Working Group
• Theory of overparametrization in quantum neural networks
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• Theory for Equivariant Quantum Neural Networks
• Recent advances for quantum classifiers
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• Layer VQE: A Variational Approach for Combinatorial Optimization on Noisy Quantum Computers
• Variational Quantum Algorithm for Estimating the Quantum Fisher Information
• Quantum simulation with hybrid tensor networks
• Does provable absence of barren plateaus imply classical simulability?
• Subtleties in the trainability of quantum machine learning models
• Analytic gradients in variational quantum algorithms: Algebraic extensions of the parameter-shift rule to general unitary transformations
• The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning
• Generalized quantum circuit differentiation rules
• Can Error Mitigation Improve Trainability of Noisy Variational Quantum Algorithms?
• Trainability barriers and opportunities in quantum generative modeling
• Absence of barren plateaus in finite local-depth circuits with long-range entanglement
• Long-time simulations with high fidelity on quantum hardware
• Dynamical simulation via quantum machine learning with provable generalization
• A semi-agnostic ansatz with variable structure for quantum machine learning
• Non-trivial symmetries in quantum landscapes and their resilience to quantum noise
• Quantum Capsule Networks
• On fundamental aspects of quantum extreme learning machines
• Entangled Datasets for Quantum Machine Learning
• Hamiltonian operator approximation for energy measurement and ground state preparation
• Retrieving information from a black hole using quantum machine learning
• Tight and Efficient Gradient Bounds for Parameterized Quantum Circuits
• Automatically Differentiable Quantum Circuit for Many-qubit State Preparation
• Efficient estimation of trainability for variational quantum circuits
• Neural network encoded variational quantum algorithms
• Numerical evidence against advantage with quantum fidelity kernels on classical data
• Quantum Convolutional Neural Networks are Effectively Classically Simulable
• DeepStability: A Study of Unstable Numerical Methods and Their Solutions in Deep Learning
• Calculating nonadiabatic couplings and Berry's phase by variational quantum eigensolvers
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• Quantum Circuit AutoEncoder
• Variational quantum solver employing the PDS energy functional
• Quantum Energy Landscape and VQA Optimization
• Assessment of the variational quantum eigensolver: application to the Heisenberg model
• Beyond unital noise in variational quantum algorithms: noise-induced barren plateaus and limit sets
• Mitigated barren plateaus in the time-nonlocal optimization of analog quantum-algorithm protocols
• Experimental Quantum Learning of a Spectral Decomposition
• The battle of clean and dirty qubits in the era of partial error correction
• Laziness, Barren Plateau, and Noise in Machine Learning
• Layering and subpool exploration for adaptive Variational Quantum Eigensolvers: Reducing circuit depth, runtime, and susceptibility to noise
• Variational quantum computing for quantum simulation: principles, implementations, and challenges
• Efficient Strategies for Reducing Sampling Error in Quantum Krylov Subspace Diagonalization
• Adaptive shot allocation for fast convergence in variational quantum algorithms
• Universal noise-precision relations in variational quantum algorithms
• Quantum computing quantum Monte Carlo algorithm
• Machine-Learning Insights into the Entanglement-trainability Correlation of Parameterized Quantum Circuits
• Efficient quantum-enhanced classical simulation for patches of quantum landscapes
• Sketching the Best Approximate Quantum Compiling Problem
• Quantum Extremal Learning
• On the similarity of bandwidth-tuned quantum kernels and classical kernels
• Universal compiling and (No-)Free-Lunch theorems for continuous variable quantum learning
• Pitfalls when tackling the exponential concentration of parameterized quantum models
• An Efficient Gradient Sensitive Alternate Framework for VQE with Variable Ansatz
• Reinforcement learning to learn quantum states for Heisenberg scaling accuracy
• Quantum Generative Training Using Rényi Divergences
• Batched Line Search Strategy for Navigating through Barren Plateaus in Quantum Circuit Training
• State preparation and evolution in quantum computing: a perspective from Hamiltonian moments
• Encoding strongly-correlated many-boson wavefunctions on a photonic quantum computer: application to the attractive Bose-Hubbard model
• The Dual Role of Low-Weight Pauli Propagation: A Flawed Simulator but a Powerful Initializer for Variational Quantum Algorithms
• Gradient Scalability and Taylor Surrogation of Quantum Cost Landscapes
• Hamiltonian Expressibility for Ansatz Selection in Variational Quantum Algorithms
• Fast suppression of classification error in variational quantum circuits
• Superior resilience to poisoning and amenability to unlearning in quantum machine learning
• Improving the trainability of VQE on NISQ computers for solving portfolio optimization using convex interpolation
• Quantum Model-Discovery
• Equivalence between exponential concentration in quantum machine learning kernels and barren plateaus in variational algorithms