163 citations · 375 across the 36 of their papers we have counts for
4 papers · 1 filter
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
Benjamin L. Edelman, Surbhi Goel, Sham Kakade +2
In modern deep learning, algorithmic choices (such as width, depth, and learning rate) are known to modulate nuanced resource tradeoffs. This work investigates how these complexiti…
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression
Adam Block, Dylan J. Foster, Akshay Krishnamurthy +2
This work studies training instabilities of behavior cloning with deep neural networks. We observe that minibatch SGD updates to the policy network during training result in sharp…
Exposing Attention Glitches with Flip-Flop Language Modeling
Bingbin Liu, Jordan T. Ash, Surbhi Goel +2
Why do large language models sometimes output factual inaccuracies and exhibit erroneous reasoning? The brittleness of these models, particularly when executing long chains of reas…
Learning Hidden Markov Models Using Conditional Samples
Sham M. Kakade, Akshay Krishnamurthy, Gaurav Mahajan +1
This paper is concerned with the computational complexity of learning the Hidden Markov Model (HMM). Although HMMs are some of the most widely used tools in sequential and time ser…