SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

Talor Abramovich, Maor Ashkenazi, Izzy Putterman +5

Speculative Decoding (SD) has emerged as a critical technique for accelerating Large Language Model (LLM) inference. Unlike deterministic system optimizations, SD performance is in…