Speculative Decoding and Beyond: An In-Depth Survey of Techniques

Yunhai Hu, Zining Liu, Zhenyuan Dong +3

Sequential dependencies present a fundamental bottleneck in deploying large-scale autoregressive models, particularly for real-time applications. While traditional optimization app…