PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation

Zihao An, Huajun Bai, Ziqiong Liu +2

The autoregressive nature of large language models (LLMs) fundamentally limits inference speed, as each forward pass generates only a single token and is often bottlenecked by memo…