PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length

Situo Zhang, Yifan Zhang, Zichen Zhu +5

Speculative decoding (SD) is a powerful technique for accelerating the inference process of large language models (LLMs) without sacrificing accuracy. Typically, SD employs a small…