Exponential convergence of testing error for stochastic gradient methods

Abstract

We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing error (classification error) converges exponentially fast if low-noise conditions are assumed.

Cited by in corpus (6)

• Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime
• A General Theory for Structured Prediction with Smooth Convex Surrogates
• Exponential Error Convergence in Data Classification with Optimized Random Features: Acceleration by Quantum Machine Learning
• Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features
• Quantifying Learning Guarantees for Convex but Inconsistent Surrogates
• Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors