56 citations · 105 across the 11 of their papers we have counts for
5 papers · 1 filter
A theory of platonic representations in language models
Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner +3
Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic representation hypothesis, yet u…
Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?
Marta Aparicio Rodriguez, Anastasia Borovykh, Grigorios A. Pavliotis +1
Generative models have a persistent limitation: their tendency to memorize training data can create legal liabilities and erode creative diversity. Understanding which samples are…
Learn from your own latents and not from tokens: A sample-complexity theory
Daniel J. Korchinski, Alessandro Favero, Matthieu Wyart
Generative models, from diffusion models to large language models, achieve remarkable performance but at a cost in training data orders of magnitude larger than what biological lea…
Sampling Data with Chains of Forward-Backward Diffusion Steps
Hyunmo Kang, Noam Itzhak Levi, Corinna Elena Wegner +2
Sampling from learned high-dimensional distributions is a foundational computational problem. We introduce U-turn chains: Markov chains obtained by iterating short forward-backward…
Symmetry in language statistics shapes the geometry of model representations
Dhruva Karkada, Daniel J. Korchinski, Andres Nava +2
The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth on…