Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion

Jacob K Christopher, Brian R Bartoldson, Tal Ben-Nun +3

Speculative decoding has emerged as a widely adopted method to accelerate large language model inference without sacrificing the quality of the model outputs. While this technique…