Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative Decoding

Jungseob Lee, Hogun Seong, Euijun Choi +1

Speculative decoding accelerates large language models by verifying drafted tokens in parallel. New draft-training objectives are usually credited with the acceptance they add over…