1 citations · 1 across the 2 of their papers we have counts for
1 paper · 1 filter
Sungkyun Kim, Jaemin Kim, Dogyung Yoon +3
LLMs have low GPU efficiency and high latency due to autoregressive decoding. Speculative decoding (SD) mitigates this using a small draft model to speculatively generate multiple…