1 citations · 2 across the 4 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse
Josh Alman, Zhao Song
Attention mechanisms lie at the heart of modern large language models (LLMs). Straightforward algorithms for forward and backward (gradient) computation take quadratic time, and a…
cs.LG2025
Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform
Josh Alman, Zhao Song
The transformer architecture has been widely applied to many machine learning tasks. A main bottleneck in the time to perform transformer computations is a task called attention co…
cs.LG2024★ 1 cited
The Fine-Grained Complexity of Gradient Computation for Training Large Language Models
Josh Alman, Zhao Song
Large language models (LLMs) have made fundamental contributions over the last a few years. To train an LLM, one needs to alternatingly run `forward' computations and `backward' co…