1 paper · 1 filter
Luxi Lin, Zhanpeng Zeng, Shuang Peng +2
Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel…