Does data interpolation contradict statistical optimality?

Abstract

We show that learning methods interpolating the training data can achieve optimal rates for the problems of nonparametric regression and prediction with square loss.

References in corpus (3)

• Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
• Rates of Convergence for Nearest Neighbor Classification
• Empirical entropy, minimax regret and minimax risk

Cited by in corpus (30)

• Reconciling modern machine learning practice and the bias-variance trade-off
• Benign Overfitting in Linear Regression
• Prevalence of Neural Collapse during the terminal phase of deep learning training
• Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
• The Role of Neural Network Activation Functions
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• Towards moderate overparameterization: global convergence guarantees for training shallow neural networks
• Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity
• Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian
• Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks
• Wide and Deep Neural Networks Achieve Optimality for Classification
• A Model of Double Descent for High-dimensional Binary Linear Classification
• Failures of model-dependent generalization bounds for least-norm interpolation
• On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels
• When Does Preconditioning Help or Hurt Generalization?
• How benign is benign overfitting?
• SGD Generalizes Better Than GD (And Regularization Doesn't Help)
• Over-parametrized deep neural networks do not generalize well
• Asymptotic theory of in-context learning by linear attention
• Interpolating Predictors in High-Dimensional Factor Regression
• On the Role of Dataset Quality and Heterogeneity in Model Confidence
• Mehler's Formula, Branching Process, and Compositional Kernels of Deep Neural Networks
• Statistical Optimality of Interpolated Nearest Neighbor Algorithms
• Interpolation and Learning with Scale Dependent Kernels
• DeepNNK: Explaining deep models and their generalization using polytope interpolation
• Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models
• Explicit Regularization of Stochastic Gradient Methods through Duality
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information
• Prediction in latent factor regression: Adaptive PCR and beyond
• How rotational invariance of common kernels prevents generalization in high dimensions