Margins, Not Windows: Training-Free Per-Step Lossy Speculative Decoding

Oszkár Urbán, Young D. Kwon, Stylianos I. Venieris +1

Speculative decoding accelerates LLM inference by drafting candidate tokens and verifying them in parallel. Tree-attention drafters such as EAGLE-3 are widely adopted, yet typicall…