1 paper
Hao Zhang, Yiming Hu, Yong Wang +3
Speculative decoding accelerates inference by using a lightweight draft model to generate candidate tokens in parallel, and are then verified by the target model, enabling lossless…