Draft, Verify, and Improve: Toward Training-Aware Speculative Decoding

Shrenik Bhansali, Larry Heck

Autoregressive (AR) decoding is a major latency bottleneck for large language models. Speculative decoding (SD) accelerates AR by letting a drafter propose multi-token blocks that…