7 citations · 9 across the 6 of their papers we have counts for
3 papers · 1 filter
DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Aozhong Zhang, Selcuk Gurses, Yanxia Deng +8
Key-value (KV) cache management has become a critical bottleneck for scaling Large Language Models (LLMs) to long-context scenarios due to linear memory growth. While recent evicti…
FlashSinkhorn 2: Block-Sparse Entropic Optimal Transport
Felix X. -F. Ye, Yu Chin Fabian Lim, Naigang Wang +1
Streaming GPU solvers for entropic optimal transport (EOT), such as FlashSinkhorn, avoid storing the dense kernel but still evaluate all point pairs in every Sinkhorn i…
XPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding
Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim +4
Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multiple-token drafting in speculative…