1 citations · 2 across the 8 of their papers we have counts for
3 papers · 1 filter
Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective
Soo Min Kwon, Alec S. Xu, Can Yaras +2
The transformer's remarkable ability to perform in-context learning (ICL) has sparked a wide range of studies designed to understand its strengths and limitations. However, a theor…
An Overview of Low-Rank Structures in the Training and Adaptation of Large Models
Laura Balzano, Tianjiao Ding, Benjamin D. Haeffele +5
The substantial computational demands of modern large-scale deep learning present significant challenges for efficient training and deployment. Recent research has revealed a wides…
Learning Dynamics of Deep Linear Networks Beyond the Edge of Stability
Avrajit Ghosh, Soo Min Kwon, Rongrong Wang +2
Deep neural networks trained using gradient descent with a fixed learning rate often operate in the regime of "edge of stability" (EOS), where the largest eigenvalue of the Hes…