Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding

Yixuan Wang, Yijun Liu, Shiyu ji +4

Large language models (LLMs) suffer from high inference latency due to the auto-regressive decoding process. Speculative decoding accelerates inference by generating multiple draft…