Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding

Bin Xiao, Lujun Gui, Lei Su +1

Large Language Models (LLMs) frequently suffer from inefficiencies, largely attributable to the discord between the requirements of auto-regressive decoding and the architecture of…