Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference
Wenjie Huang, Jun Zhang, Hongyaoxing Gu +10
Multi-turn conversations with multimodal large language models (MLLMs) reuse input context, but long multimodal inputs increase time to first token (TTFT). Query-aware hard pruning…
cs.CL2026
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?
Tianyu Liu, Yuhao Shen, Xinyi Hu +8
Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the speculative step increases. Existi…