3 papers
cs.CL2026
PA1-OPD: One-Step Prefix-Augmented On-Policy Distillation for Language Model Pretraining
Zhenhe Wu, Jian Yang, Yaping Jin +3
Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by stud…
cs.CL2026
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
Zhenhe Wu, Yaping Jin, Qinghua Xing +6
Mixture-of-Experts (MoE) models activate few experts per token, yet their full expert sets can exceed GPU memory and require repeated weight transfers during decoding. We formulate…
cs.CL2026
What Should a Skill Remember? Quality--Cost Trade-offs in Cost-Aware Skill Rewriting for Language Model Agents
Qinghua Xing, Yinda Chen, Yaping Jin +6
Large language model agents increasingly rely on skills: reusable procedural documents encoding workflows, tool use, implementation patterns, validation checks, and domain rules. S…