works on
audio-video processing 1budget allocation 1inference efficiency 1multimodal large language models 1token compression 1

From the 1 of 10 linked papers with an AI index.

activity
20242026
collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2026

When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?

Tianyu Liu, Yuhao Shen, Xinyi Hu +8

Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the speculative step increases. Existi…