5 papers · 1 filter
SARIA: Persistence-Aware KV Caching with Coordinated DRAM--SSD Reload for LLM Agents
Hui Li, Qihui Zhou, Xiao Yan +7
Multi-turn LLM agents usually reuse long context prefixes, making key value (KV) caching crucial for reducing prefill latency. SSDs provide a low-cost extension to limited DRAM cap…
QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution
Zhenduo Zhao, Qihui Zhou, Mingcong Song +7
Sparse attention reduces the cost of long-context attention, but existing kernels typically process queries independently, repeatedly loading and dequantizing KV entries shared acr…
D-Loop: Looped Diffusion Drafting for Speculative Decoding
Kecheng Chen, Yuyang He, Cheng Gong +8
Block diffusion accelerates speculative decoding by drafting multiple tokens in one forward pass. However, each position predicts a marginal distribution without observing earlier…
From Experience to Expertise: Adoption-Aware Memory Learning for Data-Scarce NPU Kernel Synthesis
Longxiao Fan, Tao Zhang, Han Yan +5
High-performance kernels underpin efficient accelerator execution but require expert tuning and lengthy manual optimization cycles. LLM coding agents promise automation, yet their…
Metis: Bridging Text and Code Memory for Self-Evolving Agents
Zijie Dai, Siuhin He, Hui Li +9
Self-evolving agents improve over time by distilling experience from past executions and reusing it in future tasks. Existing systems represent such experience either as natural-la…