Parallel Prefix Verification for Speculative Generation

Yuncheng Yao, Yuxuan Xia, Shengjie Wang +1

We introduce PARSE (PArallel pRefix Speculative Engine), a speculative generation framework that accelerates large language model (LLM) inference by parallelizing prefix verificati…