EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation

Shuyu Zhang, Lingfeng Pan, Qicheng Wang +6

Speculative decoding accelerates Large Language Model inference through draft-then-verify generation, yet lightweight draft models face coupled efficiency and quality limitations:…