2 papers
cs.CL2025
Speculative Decoding and Beyond: An In-Depth Survey of Techniques
Yunhai Hu, Zining Liu, Zhenyuan Dong +3
Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models, particularly for real-time applications. While traditional optimization app…
cs.AI2025
PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding
Bradley McDanel, Sai Qian Zhang, Yunhai Hu +1
Speculative decoding accelerates large language model inference by using smaller draft models to generate candidate tokens for parallel verification. However, current approaches ar…