Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Speculative Diffusion is a Tale of Two Tradeoffs
Aaron Zorel, Guy Van den Broeck, Aditya Grover +1
Speculative decoding is a technique that accelerates the inference speed of autoregressive large language models (LLMs) by sampling from a faster, lower quality draft model. Tradit…
cs.CL2026
A Probabilistic Interpretation of KV Cache Eviction
Renato Geh, Alex Chen, Daniel Israel +2
The premise and promise of KV (cache) eviction is simple: higher throughput can be achieved by evicting some entries from the KV cache, at a negligible cost to quality. This holds…
cs.CL2026
Probabilistic Programs of Thought
Poorva Garg, Renato Lui Geh, Daniel Israel +3
LLMs are widely used for code generation and mathematical reasoning tasks where they are required to generate structured output. They either need to reason about code, generate cod…