27 citations · 28 across the 5 of their papers we have counts for
3 papers · 1 filter
The Truth Lies Somewhere in the Middle (of the Generated Tokens)
Sophie L. Wang, Phillip Isola, Brian Cheung
How should hidden states generated autoregressively be collapsed into a representation that reflects a language model's internal state? Despite tokens being generated under causal…
The Platonic Representation Hypothesis
Minyoung Huh, Brian Cheung, Tongzhou Wang +1
We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multip…
Training Neural Networks from Scratch with Parallel Low-Rank Adapters
Minyoung Huh, Brian Cheung, Jeremy Bernstein +2
The scalability of deep learning models is fundamentally limited by computing resources, memory, and communication. Although methods like low-rank adaptation (LoRA) have reduced th…