Theory of overparametrization in quantum neural networks

Abstract

The prospect of achieving quantum advantage with Quantum Neural Networks (QNNs) is exciting. Understanding how QNN properties (e.g., the number of parameters $M$) affect the loss landscape is crucial to the design of scalable QNN architectures. Here, we rigorously analyze the overparametrization phenomenon in QNNs with periodic structure. We define overparametrization as the regime where the QNN has more than a critical number of parameters $M_c$ that allows it to explore all relevant directions in state space. Our main results show that the dimension of the Lie algebra obtained from the generators of the QNN is an upper bound for $M_c$, and for the maximal rank that the quantum Fisher information and Hessian matrices can reach. Underparametrized QNNs have spurious local minima in the loss landscape that start disappearing when $M\ge M_c$. Thus, the overparametrization onset corresponds to a computational phase transition where the QNN trainability is greatly improved by a more favorable landscape. We then connect the notion of overparametrization to the QNN capacity, so that when a QNN is overparametrized, its capacity achieves its maximum possible value. We run numerical simulations for eigensolver, compilation, and autoencoding applications to showcase the overparametrization computational phase transition. We note that our results also apply to variational quantum algorithms and quantum optimal control.

14+16 pages, 7+2 figures

References in corpus (21)

• A Quantum Approximate Optimization Algorithm
• Diagnosing Barren Plateaus with Tools from Quantum Optimal Control
• Equivalence of quantum barren plateaus to cost concentration and narrow gorges
• Capacity and quantum geometry of parametrized quantum circuits
• Fisher Information in Noisy Intermediate-Scale Quantum Applications
• Simultaneous Perturbation Stochastic Approximation of the Quantum Fisher Information
• Feedback-based quantum optimization
• Variational Hamiltonian Diagonalization for Dynamical Quantum Simulation
• Training Saturation in Layerwise Quantum Approximate Optimisation
• Progress toward favorable landscapes in quantum combinatorial optimization
• Long-time simulations with high fidelity on quantum hardware
• Searching for quantum optimal controls in the presence of singular critical points
• Learning Unitaries by Gradient Descent
• Predicting quantum dynamical cost landscapes with deep learning
• Exploiting Landscape Geometry to Enhance Quantum Optimal Control
• Entangled Datasets for Quantum Machine Learning
• Critical Points in Quantum Generative Models
• Avoiding local minima in Variational Quantum Algorithms with Neural Networks
• Best-approximation error for parametric quantum circuits
• The Inductive Bias of Quantum Kernels
• Fast Simulation of Magnetic Field Gradients for Optimization of Pulse Sequences

Cited by in corpus (92)

• Generalization in quantum machine learning from few training data
• Beyond Barren Plateaus: Quantum Variational Algorithms Are Swamped With Traps
• Barren Plateaus in Variational Quantum Computing
• A Lie Algebraic Theory of Barren Plateaus for Deep Parameterized Quantum Circuits
• Quantum Machine Learning: from physics to software engineering
• Theory for Equivariant Quantum Neural Networks
• A comprehensive review of Quantum Machine Learning: from NISQ to Fault Tolerance
• Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks
• Understanding quantum machine learning also requires rethinking generalization
• The Adjoint Is All You Need: Characterizing Barren Plateaus in Quantum Ansätze
• Generalization despite overfitting in quantum machine learning models
• Building spatial symmetries into parameterized quantum circuits for faster training
• TETRIS-ADAPT-VQE: An adaptive algorithm that yields shallower, denser circuit ansätze
• Hybrid quantum ResNet for car classification and its hyperparameter optimization
• Quantum approximate optimization via learning-based adaptive optimization
• Trainability Enhancement of Parameterized Quantum Circuits via Reduced-Domain Parameter Initialization
• An Expressive Ansatz for Low-Depth Quantum Approximate Optimisation
• Quantum Deep Hedging
• Efficient classical algorithms for simulating symmetric quantum systems
• Quantum Computational Phase Transition in Combinatorial Problems
• Nonlinear dynamics as a ground-state solution on quantum computers
• Hybrid quantum cycle generative adversarial network for small molecule generation
• Variational waveguide QED simulators
• Fourier expansion in variational quantum algorithms
• Classification of dynamical Lie algebras for translation-invariant 2-local spin systems in one dimension
• Variational quantum simulation: a case study for understanding warm starts
• Effects of noise on the overparametrization of quantum neural networks
• Training variational quantum algorithms with random gate activation
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning
• Reduction of finite sampling noise in quantum neural networks
• Tensor networks for interpretable and efficient quantum-inspired machine learning
• Generalization of Quantum Machine Learning Models Using Quantum Fisher Information Metric
• A General Approach to Dropout in Quantum Neural Networks
• Lie-algebraic classical simulations for quantum computing
• Here comes the SU(N): multivariate quantum gates and gradients
• Quantum neural networks form Gaussian processes
• Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Series
• What can we learn from quantum convolutional neural networks?
• Statistical Complexity of Quantum Learning
• Diabatic Quantum Annealing for the Frustrated Ring Model
• EHA: Entanglement-variational Hardware-efficient Ansatz for Eigensolvers
• Optimizing Variational Quantum Algorithms with qBang: Efficiently Interweaving Metric and Momentum to Navigate Flat Energy Landscapes
• Constrained and Vanishing Expressivity of Quantum Fourier Models
• Expressivity of Variational Quantum Machine Learning on the Boolean Cube
• Fermionic Simulators for Enhanced Scalability of Variational Quantum Simulation
• Trainability Barriers in Low-Depth QAOA Landscapes
• Splitting and Parallelizing of Quantum Convolutional Neural Networks for Learning Translationally Symmetric Data
• Numerical solution of nonlinear Schrödinger equation by a hybrid pseudospectral-variational quantum algorithm
• Barren plateaus are swamped with traps
• Image Classification with Rotation-Invariant Variational Quantum Circuits
• Random coordinate descent: a simple alternative for optimizing parameterized quantum circuits
• Protocols for Trainable and Differentiable Quantum Generative Modelling
• Variational Quantum Circuit Design for Quantum Reinforcement Learning on Continuous Environments
• SnCQA: A hardware-efficient equivariant quantum convolutional circuit architecture
• Photovoltaic power forecasting using quantum machine learning
• Gradients and frequency profiles of quantum re-uploading models
• Analyzing the quantum approximate optimization algorithm: ansätze, symmetries, and Lie algebras
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits
• Framework for Learning and Control in the Classical and Quantum Domains
• Compilation of Entangling Gates for High-Dimensional Quantum Systems
• Efficiency of neural quantum states in light of the quantum geometric tensor
• Lorentz-Equivariant Quantum Graph Neural Network for High-Energy Physics
• Benchmarking hybrid digitized-counterdiabatic quantum optimization
• Trade-off between Gradient Measurement Efficiency and Expressivity in Deep Quantum Neural Networks
• Characterization of overparametrization in the simulation of realistic quantum systems
• Energy Landscape Plummeting in Variational Quantum Eigensolver: Subspace Optimization, Non-iterative Corrections and Generator-informed Initialization for Improved Quantum Efficiency
• Scalability Challenges in Variational Quantum Optimization under Stochastic Noise
• Variational Optimization for Quantum Problems using Deep Generative Networks
• Classification and reconstruction for single-pixel imaging with classical and quantum neural networks
• Supervised Learning Guarantee for Quantum AdaBoost
• Quantum subspace expansion approach for simulating dynamical response functions of Kitaev spin liquids
• Non-Iterative Disentangled Unitary Coupled-Cluster based on Lie-algebraic structure
• Parallel-in-time quantum simulation via Page and Wootters quantum time
• Data-Dependent Generalization Bounds for Parameterized Quantum Models Under Noise
• Architectures and random properties of symplectic quantum circuits
• Trained quantum neural networks are Gaussian processes
• Pitfalls when tackling the exponential concentration of parameterized quantum models
• Cross-platform hardware benchmark of style-based quantum GANs for data augmentation on superconducting and trapped-ion processors
• The Lie Algebra of XY-mixer Topologies and Warm Starting QAOA for Constrained Optimization
• Adiabatic quantum computing with parameterized quantum circuits
• Global optimization in variational quantum algorithms via dynamic tunneling method
• OnionVQE Optimization Strategy for Ground State Preparation on NISQ Devices
• Quantum-data-driven dynamical transition in quantum learning
• Double descent in quantum kernel methods
• Expressiveness of Commutative Quantum Circuits: A Probabilistic Approach
• Generative flow-based warm start of the variational quantum eigensolver
• Multi-objective optimization and quantum hybridization of equivariant deep learning interatomic potentials
• Dynamical transition in controllable quantum neural networks with large depth
• Role of overparametrization in quantum approximate optimization
• Disentangling quantum autoencoder
• Gradient Scalability and Taylor Surrogation of Quantum Cost Landscapes
• Probabilistic Representation of Commutative Quantum Circuit Models