170 citations · 177 across the 12 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
Linearizing Large Language Models
Jean Mercat, Igor Vasiljevic, Sedrick Keh +4
Linear transformers have emerged as a subquadratic-time alternative to softmax attention and have garnered significant interest due to their fixed-size recurrent state that lowers…
cs.CL2024
Language models scale reliably with over-training and on downstream tasks
Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar +22
Scaling laws are useful guides for derisking expensive training runs, as they predict performance of large models using cheaper, small-scale experiments. However, there remain gaps…