24 citations · 87 across the 13 of their papers we have counts for
3 papers · 1 filter
When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting
Vedant Palit, Florent Draye, Nicolas Zucchet +2
Knowledge that a language model appears to forget during finetuning often remains stored and can be recovered, a phenomenon called spurious forgetting. Finetuning on new facts can…
Language models suffer from a curse of ambiguity
Nicolas Zucchet, Hyun Dong Lee, Scott Linderman
Large language models increasingly rely on sampling as a driver of their own improvement, making the fidelity of their learned distributions more critical than ever. Yet, not all d…
How do language models learn facts? Dynamics, curricula and hallucinations
Nicolas Zucchet, Jörg Bornschein, Stephanie Chan +3
Large language models accumulate vast knowledge during pre-training, yet the dynamics governing this acquisition remain poorly understood. This work investigates the learning dynam…