Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Compute-Optimal LLMs Provably Generalize Better With Scale
Marc Finzi, Sanyam Kapoor, Diego Granziol +4
Why do larger language models generalize better? To investigate this question, we develop generalization bounds on the pretraining objective of large language models (LLMs) in the…
cs.LG2024
Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
Andres Potapczynski, Shikai Qiu, Marc Finzi +6
Dense linear layers are the dominant computational bottleneck in large neural networks, presenting a critical need for more efficient alternatives. Previous efforts focused on a sm…
cs.LG2024
STAT: Shrinking Transformers After Training
Megan Flynn, Alexander Wang, Dean Edward Alvarez +2
We present STAT: a simple algorithm to prune transformer models without any fine-tuning. STAT eliminates both attention heads and neurons from the network, while preserving accurac…