4 papers
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Yu-Yang Qian, Hao-Cong Wu, Chen Chen +4
Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for…
JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting
Lanxiang Hu, Zhaoxiang Feng, Yulun Wu +9
Speculative decoding (SD) accelerates autoregressive Large Language Models (LLMs) by drafting multiple tokens and verifying them in parallel, but it faces a scaling limitation: inc…
When Drafts Evolve: Speculative Decoding Meets Online Learning
Yu-Yang Qian, Hao-Cong Wu, Yichao Fu +2
Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens th…
ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp
Xinhang Chen, Chao Zhang, Jiahuan He +9
DeepSeek-V3.2-Exp introduces a sparse attention mechanism that significantly reduces inference latency in long-context scenarios. Although the overall throughput has improved great…