1 paper · 1 filter
Taehyeon Kim, Hojung Jung, Se-Young Yun
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens, which are then verified by the target LLM. This preserve…