16 papers
LOCAL: Enabling Learning On-device Contiguously for Agent LLMs
Xinxin Liu, Jiaxin Li, Zibo Wang +7
On-device LLM agents interact repeatedly with users on local hardware, producing private traces that are valuable for adaptation but should not be sent to a remote trainer. Ideally…
TIDE-MC: Two-Sided Interpolative Decomposition for Billion-Scale GPU Matrix Completion
Chengying Huan, Yubo Wang, Pinhuan Wang +11
Matrix completion supports large-scale recommendation and scientific computing, yet existing GPU solvers commonly assume that the observed matrix or its dense factors fit in device…
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
Zhibin Wang, Xuying Han, Zhaohua Yang +5
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculati…
SmartSwap: Swap-Based Memory Optimization for LLM Training under Varying Operator Sequences
Zibo Wang, Yuhang Zhou, Zhibin Wang +13
The paper introduces SmartSwap, a swap‑based memory optimization framework that adapts to varying operator sequences in eager‑mode training of large language models, using a lightw…
CacheWeaver: Cache-Aware Evidence Ordering for Efficient Grounded RAG Inference
Kaizhen Tan, Rong Gu, Mingyuan Li
Retrieval-Augmented Generation (RAG) improves factual grounding, but it also lengthens prompts and raises prefill cost. Prefix caching in serving engines such as vLLM reduces this…
SSV: Sparse Speculative Verification for Efficient LLM Inference
Zhibin Wang, Ziyu Zhong, Nuo Shen +3
Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across m…