101 citations · 158 across the 12 of their papers we have counts for
3 papers · 1 filter
OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
Jaeseong Lee, seung-won hwang, Aurick Qiao +3
Speculative decoding promises faster inference for large language models (LLMs), yet existing methods fail to generalize to real-world settings. Benchmarks typically assume short c…
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Rui Pan, Yinwei Dai, Zhihao Zhang +3
Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs).…
AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
Zikun Li, Zhuofu Chen, Remi Delacourt +11
Modern large language model (LLM) applications exhibit diverse service-level objectives (SLOs), from low-latency requirements in interactive coding assistants to more relaxed const…