290 citations · 1.8k across the 59 of their papers we have counts for
3 papers · 1 filter
The emergence of clusters in self-attention dynamics
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy +1
Viewing Transformers as interacting particle systems, we describe the geometry of learned representations when the weights are not time dependent. We show that particles, represent…
GULP: a prediction-based metric between representations
Enric Boix-Adsera, Hannah Lawrence, George Stepaniants +1
Comparing the representations learned by different neural networks has recently emerged as a key tool to understand various architectures and ultimately optimize them. In this work…
Rejection sampling from shape-constrained distributions in sublinear time
Sinho Chewi, Patrik Gerber, Chen Lu +2
We consider the task of generating exact samples from a target distribution, known up to normalization, over a finite alphabet. The classical algorithm for this task is rejection s…