10 papers
MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression
Sheng Qiang, Ruiwei Chen, Yinpeng Wu +5
Long-context LLM services now sustain prompts with hundreds of thousands to millions of tokens, making the key-value (KV) cache a first-order serving cost. Because the cache grows…
DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
Yunpeng Dong, Jingkai He, Shiqi Liu +7
LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the compl…
DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
Zhenliang Xue, Hanpeng Hu, Xing Chen +7
Large multimodal models (LMMs) have demonstrated excellent capabilities in both understanding and generation tasks with various modalities. While these models can accept flexible c…
Sharpen the Spec, Cut the Code: A Case for Generative File System with SYSSPEC
Qingyuan Liu, Mo Zou, Hengbin Zhang +3
File systems are critical OS components that require constant evolution to support new hardware and emerging application needs. However, the traditional paradigm of developing feat…
Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
Le Chen, Dahu Feng, Erhu Feng +5
With the rapid advancement of artificial intelligence technologies such as ChatGPT, AI agents, and video generation, contemporary mobile systems have begun integrating these AI cap…
History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
Jingkai He, Tianjian Li, Erhu Feng +5
With the rapid advancement of large language models (LLMs), reinforcement learning (RL) has emerged as a pivotal methodology for enhancing the reasoning capabilities of LLMs. Unlik…