Laziness, Barren Plateau, and Noise in Machine Learning

Abstract

We define \emph{laziness} to describe a large suppression of variational parameter updates for neural networks, classical or quantum. In the quantum case, the suppression is exponential in the number of qubits for randomized variational quantum circuits. We discuss the difference between laziness and \emph{barren plateau} in quantum machine learning created by quantum physicists in \cite{mcclean2018barren} for the flatness of the loss function landscape during gradient descent. We address a novel theoretical understanding of those two phenomena in light of the theory of neural tangent kernels. For noiseless quantum circuits, without the measurement noise, the loss function landscape is complicated in the overparametrized regime with a large number of trainable variational angles. Instead, around a random starting point in optimization, there are large numbers of local minima that are good enough and could minimize the mean square loss function, where we still have quantum laziness, but we do not have barren plateaus. However, the complicated landscape is not visible within a limited number of iterations, and low precision in quantum control and quantum sensing. Moreover, we look at the effect of noises during optimization by assuming intuitive noise models, and show that variational quantum algorithms are noise-resilient in the overparametrization regime. Our work precisely reformulates the quantum barren plateau statement towards a precision statement and justifies the statement in certain noise models, injects new hope toward near-term variational quantum algorithms, and provides theoretical connections toward classical machine learning. Our paper provides conceptual perspectives about quantum barren plateaus, together with discussions about the gradient descent dynamics in \cite{together}.

18 pages, 3 figures

References in corpus (19)

• Variational Quantum Algorithms
• A Quantum Approximate Optimization Algorithm
• Black holes as mirrors: quantum information in random subsystems
• Noise-Induced Barren Plateaus in Variational Quantum Algorithms
• A rigorous and robust quantum speed-up in supervised machine learning
• Absence of Barren Plateaus in Quantum Convolutional Neural Networks
• Chaos, Complexity, and Random Matrices
• The Principles of Deep Learning Theory
• Effect of barren plateaus on gradient-free optimization
• Optimal Quantum Measurements of Expectation Values of Observables
• Higher Order Derivatives of Quantum Neural Networks with Barren Plateaus
• Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
• Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks
• Representation Learning via Quantum Neural Tangent Kernels
• Neural Networks and Quantum Field Theory
• Analytic theory for the dynamics of wide quantum neural networks
• Asymptotics of Wide Networks from Feynman Diagrams
• Feature Learning in Infinite-Width Neural Networks
• A Convergence Theory for Over-parameterized Variational Quantum Eigensolvers

Cited by in corpus (7)

• Barren Plateaus in Variational Quantum Computing
• Stochastic noise can be helpful for variational quantum algorithms
• Variational-quantum-eigensolver-inspired optimization for spin-chain work extraction
• Quantum Physics-Informed Neural Networks for Maxwell's Equations: Circuit Design, "Black Hole" Barren Plateaus Mitigation, and GPU Acceleration
• Quantum-data-driven dynamical transition in quantum learning
• Quantum Bayesian Networks for Machine Learning in Oil-Spill Detection
• Role of overparametrization in quantum approximate optimization