212 citations · 457 across the 29 of their papers we have counts for
5 papers · 1 filter
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
Jingfeng Wu, Difan Zou, Zixiang Chen +3
Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities, enabling them to solve unseen tasks solely based on input contexts without adjus…
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
Yuan Cao, Difan Zou, Yuanzhi Li +1
We study the implicit bias of batch normalization trained by gradient descent. We show that when learning a linear model with batch normalization for binary classification, gradien…
Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
Xuran Meng, Yuan Cao, Difan Zou
Gradient regularization, as described in \citet{barrett2021implicit}, is a highly effective technique for promoting flat minima during gradient descent. Empirical evidence suggests…
The Benefits of Mixup for Feature Learning
Difan Zou, Yuan Cao, Yuanzhi Li +1
Mixup, a simple data augmentation method that randomly mixes two data points via linear interpolation, has been extensively applied in various deep learning applications to gain be…
Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuron
Jingfeng Wu, Difan Zou, Zixiang Chen +3
This paper considers the problem of learning a single ReLU neuron with squared loss (a.k.a., ReLU regression) in the overparameterized regime, where the input dimension can exceed…