Recurrent Drafter for Fast Speculative Decoding in Large Language Models

Yunfei Cheng, Aonan Zhang, Xuanyu Zhang +2

We present Recurrent Drafter (ReDrafter), an advanced speculative decoding approach that achieves state-of-the-art speedup for large language models (LLMs) inference. The performan…