1 citations · 1 across the 21 of their papers we have counts for
9 papers · 1 filter
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
Naibin Gu, Qingyi Si, Chenxu Yang +5
On-policy distillation (OPD) provides dense token-level supervision on student-generated trajectories. Although OPD performs strongly when teacher and student belong to the same mo…
CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
Runze Liu, Naibin Gu, Mingxu Ai +4
Continual multimodal instruction tuning requires multimodal large language models to acquire new task abilities sequentially while preserving previously learned knowledge. LoRA-MoE…
FOVEA: Focused On-Demand Visual Evidence Adaptation for Cache-Friendly Multimodal Speculative Decoding
Hengjie Zhu, Dayan Wu, Zihao Zhang +6
Multimodal speculative decoding accelerates vision-language models by allowing a lightweight draft model to propose candidate tokens for parallel verification by a larger target mo…
Harnessing Streaming Video in the Wild
Dingyu Yao, Shuhuan Gu, Qingyi Si +8
Vision-Language Models (VLMs) are increasingly required to process unbounded video streams in applications such as video-call assistants, live commentary, and embodied robots. An i…
Co-Evolving Policy Distillation
Naibin Gu, Chenxu Yang, Qingyi Si +7
RLVR and OPD have become standard paradigms for post-training. We provide a unified analysis of these two paradigms in consolidating multiple expert capabilities into a single mode…
Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?
Yatong Wang, Fali Wang, Naibin Gu +6
Cloud-hosted large language models (LLMs) commonly rely on LoRA for domain adaptation, yet domain data are distributed across multiple edge devices and cannot be uploaded due to pr…
Self-Distilled RLVR
Chenxu Yang, Chuanyu Qin, Qingyi Si +7
On-policy distillation (OPD) has become a popular training paradigm in the LLM community. This paradigm selects a larger model as the teacher to provide dense, fine-grained signals…
Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring
Weixin Guan, Liang Li, Jiapeng Liu +6
Large Reasoning Language Models (LRLMs) demonstrate impressive capabilities on complex tasks by utilizing long Chain-of-Thought reasoning. However, they are prone to overthinking,…
System 1&2 Synergy via Dynamic Model Interpolation
Chenxu Yang, Qingyi Si, Chong Tian +6
Training a unified language model that adapts between intuitive System 1 and deliberative System 2 remains challenging due to interference between their cognitive modes. Recent stu…