Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge

Bin Xiao, Chunan Shi, Xiaonan Nie +5

Large language models (LLMs) suffer from low efficiency as the mismatch between the requirement of auto-regressive decoding and the design of most contemporary GPUs. Specifically,…