306 citations · 371 across the 10 of their papers we have counts for
3 papers · 1 filter
Scalable Optimization in the Modular Norm
Tim Large, Yang Liu, Minyoung Huh +3
To improve performance in contemporary deep learning, one is interested in scaling up the neural network in terms of both the number and the size of the layers. When ramping up the…
The Platonic Representation Hypothesis
Minyoung Huh, Brian Cheung, Tongzhou Wang +1
We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multip…
Training Neural Networks from Scratch with Parallel Low-Rank Adapters
Minyoung Huh, Brian Cheung, Jeremy Bernstein +2
The scalability of deep learning models is fundamentally limited by computing resources, memory, and communication. Although methods like low-rank adaptation (LoRA) have reduced th…