A Practical Investigation of Training-free Relaxed Speculative Decoding

Guoxuan Xia, Luka Ribar, Paul Balanca

Speculative decoding accelerates sampling from an autoregressive LLM by using a faster auxiliary model to draft tokens which are then verified in parallel by the LLM. Standard spec…