1 citations · 5 across the 15 of their papers we have counts for
9 papers · 1 filter
Internal Guidance as a Sampling Method for Tilted Diffusion Models
Curtis McDonald, Peter Bartlett
Reward guidance methods sample an exponential tilt of a base density using a diffusion model trained with only samples from the base distribution. Existing methods use what we call…
Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression
Juno Kim, Hengyu Fu, Peter Bartlett +2
We compare the instance-wise, finite-sample risks of monotone spectral filters for linear regression, a broad class of estimators including principal component regression (PCR), gr…
Hard labels sampled from sparse targets mislead rotation invariant algorithms
Avrajit Ghosh, Bin Yu, Manfred Warmuth +1
One of the most common machine learning setups is logistic regression. In many classification models, including neural networks, the final prediction is obtained by applying a logi…
Risk Comparisons in Linear Regression: Implicit Regularization Dominates Explicit Regularization
Jingfeng Wu, Peter L. Bartlett, Sham M. Kakade +2
Existing theory suggests that for linear regression problems categorized by capacity and source conditions, gradient descent (GD) is always minimax optimal, while both ridge regres…
Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
Jingfeng Wu, Pierre Marion, Peter Bartlett
We study gradient descent (GD) with a constant stepsize for -regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensu…
Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes
Ruiqi Zhang, Jingfeng Wu, Licong Lin +1
We study (GD) for logistic regression on linearly separable data with stepsizes that adapt to the current risk, scaled by a constant hyperparameter .…
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
Yuhang Cai, Jingfeng Wu, Song Mei +2
The typical training of neural networks using large stepsize gradient descent (GD) under the logistic loss often involves two distinct phases, where the empirical risk oscillates i…
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett
We study the \emph{in-context learning} (ICL) ability of a \emph{Linear Transformer Block} (LTB) that combines a linear attention component and a linear multi-layer perceptron (MLP…
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
Jingfeng Wu, Difan Zou, Zixiang Chen +3
Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities, enabling them to solve unseen tasks solely based on input contexts without adjus…