Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning

Jiebin Zhang, Zhenghan Yu, Liang Wang +8

Speculative decoding accelerates large language model (LLM) inference by using a small draft model to generate candidate tokens for a larger target model to verify. The efficacy of…