2 papers
cs.LG2026
PulseInfer: I/O-Centric Sparse KV Cache Offloading for Efficient Long-Context LLM Decoding
Qiuyang Zhang, Kai Zhou, Kai Lu +6
Long-context LLM serving is increasingly bottlenecked by decode, where large KV caches limit batch size and underutilize GPUs. Sparse KV cache offloading expands effective capacity…
cs.CL2026
GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring
Xuming Ye, Zeming Ma, Runjie Yu +5
Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree builders typically construct t…