4 papers
MentorPulse: Refreshing Cross-Model Latent Guidance for Long-Form Generation
Ziwu Liu, Guozhong Li, Chen Qiu +2
Cross-model latent guidance lets a frozen large mentor encode an input once and a frozen small student generate from the resulting signal. Existing methods keep this signal fixed,…
KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
Chen Qiu, Ziwu Liu, Chao Fei +2
KV-cache compression reduces long-context memory, but aggregate task scores reveal neither which correct executions fail nor why. We present KVDiagnosis, a diagnostic dataset and b…
AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
Chengyu Qiu, Xiao Fu, Fengcun Li +6
Attention--Feed-Forward Network (FFN) Disaggregation (AFD) is emerging as a promising architecture for serving Mixture-of-Experts (MoE) language models. While existing AFD systems…
ART: Attention Run-time Termination for Efficient Large Language Model Decoding
Chen Qiu, Guozhong Li, Cristian McGee +2
Long-context decoding in Large Language Models (LLMs) is constrained by the cost of accessing and processing the Key-Value (KV) cache. Despite evidence that attention outputs depen…