1 citations · 5 across the 15 of their papers we have counts for
Showing 2024 · stat.MLShow all
2 papers · 2 filters
stat.ML2024
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
Yuhang Cai, Jingfeng Wu, Song Mei +2
The typical training of neural networks using large stepsize gradient descent (GD) under the logistic loss often involves two distinct phases, where the empirical risk oscillates i…
stat.ML2024★ 1 cited
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization
Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett
We study the \emph{in-context learning} (ICL) ability of a \emph{Linear Transformer Block} (LTB) that combines a linear attention component and a linear multi-layer perceptron (MLP…