11 papers
LOCAL: Enabling Learning On-device Contiguously for Agent LLMs
Xinxin Liu, Jiaxin Li, Zibo Wang +7
On-device LLM agents interact repeatedly with users on local hardware, producing private traces that are valuable for adaptation but should not be sent to a remote trainer. Ideally…
MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training
Yikai Wang, Chuansai Zhou, Yuhang Zhou +10
Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead. In…
SpecLA: Efficient Speculative Decoding for Linear-Attention Models
Zhibin Wang, Xuying Han, Zhaohua Yang +5
Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculati…
SSV: Sparse Speculative Verification for Efficient LLM Inference
Zhibin Wang, Ziyu Zhong, Nuo Shen +3
Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across m…
STAR: Decode-Phase Rescheduling for LLM Inference
Zhibin Wang, Zetao Hong, Xue Li +8
Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalance in the decode phase, particularly f…
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…