Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

Zipeng Gao, Zhi Zheng, Qingrong Xia +5

Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks. However, traditional speculative decoding typically…