4 papers
Speculative Diffusion is a Tale of Two Tradeoffs
Aaron Zorel, Guy Van den Broeck, Aditya Grover +1
Speculative decoding is a technique that accelerates the inference speed of autoregressive large language models (LLMs) by sampling from a faster, lower quality draft model. Tradit…
Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectives
Hua, Xu, Dongxin Li +4
Discrete diffusion models generate sequences by iteratively resolving multiple tokens in parallel, offering a flexible alternative to left-to-right generation. However, guiding thi…
A Probabilistic Interpretation of KV Cache Eviction
Renato Geh, Alex Chen, Daniel Israel +2
The premise and promise of KV (cache) eviction is simple: higher throughput can be achieved by evicting some entries from the KV cache, at a negligible cost to quality. This holds…
On the Complexity and Approximation of Binary Evidence in Lifted Inference
Guy Van den Broeck, Adnan Darwiche
Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relat…