105 citations · 151 across the 5 of their papers we have counts for
5 papers
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
Pascal Jr. Tikeng Notsawo, Hattie Zhou, Mohammad Pezeshki +2
This paper focuses on predicting the occurrence of grokking in neural networks, a phenomenon in which perfect generalization emerges long after signs of overfitting or memorization…
Teaching Algorithmic Reasoning via In-context Learning
Hattie Zhou, Azade Nova, Hugo Larochelle +3
Large language models (LLMs) have shown increasing in-context learning capabilities through scaling up model and data size. Despite this progress, LLMs are still unable to solve al…
Fortuitous Forgetting in Connectionist Networks
Hattie Zhou, Ankit Vani, Hugo Larochelle +1
Forgetting is often seen as an unwanted characteristic in both human and machine learning. However, we propose that forgetting can in fact be favorable to learning. We introduce "f…
LCA: Loss Change Allocation for Neural Network Training
Janice Lan, Rosanne Liu, Hattie Zhou +1
Neural networks enjoy widespread use, but many aspects of their training, representation, and operation are poorly understood. In particular, our view into the training process is…
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
Hattie Zhou, Janice Lan, Rosanne Liu +1
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are tr…