5 papers
Improved Scaling Laws in Linear Regression via Data Reuse
Licong Lin, Jingfeng Wu, Peter L. Bartlett
Neural scaling laws suggest that the test error of large language models trained online decreases polynomially as the model size and data size increase. However, such scaling can b…
Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
Jingfeng Wu, Pierre Marion, Peter Bartlett
We study gradient descent (GD) with a constant stepsize for -regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensu…
Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes
Ruiqi Zhang, Jingfeng Wu, Licong Lin +1
We study (GD) for logistic regression on linearly separable data with stepsizes that adapt to the current risk, scaled by a constant hyperparameter .…
Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
Yuhang Cai, Kangjie Zhou, Jingfeng Wu +3
We establish the asymptotic implicit bias of gradient descent (GD) for generic non-homogeneous deep networks under exponential loss. Specifically, we characterize three key propert…
Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression
Jingfeng Wu, Peter Bartlett, Matus Telgarsky +1
In overparameterized logistic regression, gradient descent (GD) iterates diverge in norm while converging in direction to the maximum -margin solution -- a phenomenon known…