563 citations · 6.7k across the 119 of their papers we have counts for
124 papers
A theory of platonic representations in language models
Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner +3
Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic representation hypothesis, yet u…
Learn from your own latents and not from tokens: A sample-complexity theory
Daniel J. Korchinski, Alessandro Favero, Matthieu Wyart
Generative models, from diffusion models to large language models, achieve remarkable performance but at a cost in training data orders of magnitude larger than what biological lea…
Sampling Data with Chains of Forward-Backward Diffusion Steps
Hyunmo Kang, Noam Itzhak Levi, Corinna Elena Wegner +2
Sampling from learned high-dimensional distributions is a foundational computational problem. We introduce U-turn chains: Markov chains obtained by iterating short forward-backward…
Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence
Andres Nava, Matthieu Wyart
We propose a distributional theory of how hypernymy -- the ``is-a'' relation between general and specific concepts -- is encoded geometrically in language representations. Starting…
Theories of the Glass Transition Based on Local Excitations
Massimo Pica Ciamarra, Jeppe C. Dyre, Edan Lerner +1
The dramatic slowdown of dynamics in supercooled liquids approaching the glass transition remains one of the central unresolved problems in condensed matter physics. We review appr…
Symmetry in language statistics shapes the geometry of model representations
Dhruva Karkada, Daniel J. Korchinski, Andres Nava +2
The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth on…