Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding

Jun Zhang, Jue Wang, Huan Li +4

We present a novel inference scheme, self-speculative decoding, for accelerating Large Language Models (LLMs) without the need for an auxiliary model. This approach is characterize…