1 citations · 2 across the 5 of their papers we have counts for
7 papers
Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization
Jingfeng Wu, Peter L. Bartlett, Sham M. Kakade +2
Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regres…
Improved Scaling Laws in Linear Regression via Data Reuse
Licong Lin, Jingfeng Wu, Peter L. Bartlett
Neural scaling laws suggest that the test error of large language models trained online decreases polynomially as the model size and data size increase. However, such scaling can b…
Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
Jingfeng Wu, Pierre Marion, Peter Bartlett
We study gradient descent (GD) with a constant stepsize for -regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensu…
Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes
Ruiqi Zhang, Jingfeng Wu, Licong Lin +1
We study (GD) for logistic regression on linearly separable data with stepsizes that adapt to the current risk, scaled by a constant hyperparameter .…
Context-Scaling versus Task-Scaling in In-Context Learning
Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1
Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
Yuhang Cai, Jingfeng Wu, Song Mei +2
The typical training of neural networks using large stepsize gradient descent (GD) under the logistic loss often involves two distinct phases, where the empirical risk oscillates i…
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett
We study the \emph{in-context learning} (ICL) ability of a \emph{Linear Transformer Block} (LTB) that combines a linear attention component and a linear multi-layer perceptron (MLP…