1 paper · 1 filter
Roy Eisenstadt, Harel Lidar, Tal Gonen +5
Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target m…