102 citations · 280 across the 26 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
Efficient Stagewise Pretraining via Progressive Subnetworks
Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu +4
Recent developments in large language models have sparked interest in efficient pretraining methods. Stagewise training approaches to improve efficiency, like gradual stacking and…
cs.CL2023★ 4 cited
DistillSpec: Improving Speculative Decoding via Knowledge Distillation
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat +5
Speculative decoding (SD) accelerates large language model inference by employing a faster draft model for generating multiple tokens, which are then verified in parallel by the la…