Generalization despite overfitting in quantum machine learning models

Abstract

The widespread success of deep neural networks has revealed a surprise in classical machine learning: very complex models often generalize well while simultaneously overfitting training data. This phenomenon of benign overfitting has been studied for a variety of classical models with the goal of better understanding the mechanisms behind deep learning. Characterizing the phenomenon in the context of quantum machine learning might similarly improve our understanding of the relationship between overfitting, overparameterization, and generalization. In this work, we provide a characterization of benign overfitting in quantum models. To do this, we derive the behavior of a classical interpolating Fourier features models for regression on noisy signals, and show how a class of quantum models exhibits analogous features, thereby linking the structure of quantum circuits (such as data-encoding and state preparation operations) to overparameterization and overfitting in quantum models. We intuitively explain these features according to the ability of the quantum model to interpolate noisy data with locally "spiky" behavior and provide a concrete demonstration example of benign overfitting.

arXiv comment47 pages, 6 figures

References in corpus (35)

• Supervised learning with quantum enhanced feature spaces
• Barren plateaus in quantum neural network training landscapes
• Quantum machine learning in feature Hilbert spaces
• Quantum Circuit Learning
• Reconciling modern machine learning practice and the bias-variance trade-off
• Parameterized quantum circuits as machine learning models
• Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits
• Evaluating analytic gradients on quantum hardware
• Circuit-centric quantum classifiers
• The power of quantum neural networks
• Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms
• Noise-Induced Barren Plateaus in Variational Quantum Algorithms
• The effect of data encoding on the expressive power of variational quantum machine learning models
• Power of data in quantum machine learning
• Data re-uploading for a universal quantum classifier
• Connecting ansatz expressibility to gradient magnitudes and barren plateaus
• Quantum generative adversarial learning
• Benign Overfitting in Linear Regression
• Quantum advantage in learning from experiments
• Quantum generative adversarial networks
• Generalization in quantum machine learning from few training data
• Chaos and complexity by design
• Quantum Simulation of Electronic Structure with Linear Depth and Connectivity
• The Expressive Power of Parameterized Quantum Circuits
• General parameter-shift rules for quantum gradients
• Solving strongly correlated electron models on a quantum computer
• Information-theoretic bounds on quantum advantage in machine learning
• Quantum machine learning beyond kernel methods
• Exploiting symmetry in variational quantum machine learning
• Theory of overparametrization in quantum neural networks
• Group-Invariant Quantum Machine Learning
• Generalization in Quantum Machine Learning: a Quantum Information Perspective
• Symbolic integration with respect to the Haar measure on the unitary group
• Structural risk minimization for quantum linear classifiers
• Exponential data encoding for quantum supervised learning

Cited by in corpus (27)

• Understanding quantum machine learning also requires rethinking generalization
• Benchmarking quantum machine learning kernel training for classification tasks
• Parallel Hybrid Networks: an interplay between quantum and classical neural networks
• Let Quantum Neural Networks Choose Their Own Frequencies
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning
• Generalization of Quantum Machine Learning Models Using Quantum Fisher Information Metric
• Dequantizing quantum machine learning models using tensor networks
• A General Approach to Dropout in Quantum Neural Networks
• Training robust and generalizable quantum models
• An exponentially-growing family of universal quantum circuits
• Constrained and Vanishing Expressivity of Quantum Fourier Models
• Satellite image classification with neural quantum kernels
• Photovoltaic power forecasting using quantum machine learning
• Enhanced feature encoding and classification on distributed quantum hardware
• Gradients and frequency profiles of quantum re-uploading models
• A Post-Training Approach for Mitigating Overfitting in Quantum Convolutional Neural Networks
• Quantum neural compressive sensing for ghost imaging
• Quantum Circuit Design using a Progressive Widening Enhanced Monte Carlo Tree Search
• Learning Fourier series with parametrized quantum circuits
• Fourier Analysis of Variational Quantum Circuits for Supervised Learning
• Towards Improved Quantum Machine Learning for Molecular Force Fields
• When Quantum and Classical Models Disagree: Learning Beyond Minimum Norm Least Square
• Double descent in quantum kernel methods
• Benchmarking Quantum Surrogate Models on Scarce and Noisy Data
• Quantum generative classification with mixed states
• Hybrid quantum recurrent neural network for remaining useful life prediction of turbofan engines
• Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems