4 papers
Optimizers Qualitatively Alter Solutions And We Should Leverage This
Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu +6
Due to the nonlinear nature of Deep Neural Networks (DNNs), one can not guarantee convergence to a unique global minimum of the loss when using optimizers relying only on local inf…
Scaling Human Judgment in Community Notes with LLMs
Haiwen Li, Soham De, Manon Revel +6
This paper argues for a new paradigm for Community Notes in the LLM era: an open ecosystem where both humans and LLMs can write notes, and the decision of which notes are helpful e…
From job titles to jawlines: Using context voids to study generative AI systems
Shahan Ali Memon, Soham De, Sungha Kang +5
In this paper, we introduce a speculative design methodology for studying the behavior of generative AI systems, framing design as a mode of inquiry. We propose bridging seemingly…
How do language models learn facts? Dynamics, curricula and hallucinations
Nicolas Zucchet, Jörg Bornschein, Stephanie Chan +3
Large language models accumulate vast knowledge during pre-training, yet the dynamics governing this acquisition remain poorly understood. This work investigates the learning dynam…