1 paper · 2 filters
Luxi Lin, Zhanpeng Zeng, Shuang Peng +2
Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel…