14 citations · 14 across the 6 of their papers we have counts for
7 papers
Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
Yuanyi Wang, Yanggan Gu, Su Lu +7
Model merging efficiently combines specialized large language models (LLMs) without joint retraining, but can substantially alter expert routing in Mixture-of-Experts (MoE) models.…
Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
Yuanyi Wang, Su Lu, Yanggan Gu +6
On-policy distillation (OPD) trains a student on its own rollouts with token-level teacher supervision. Recent selective OPD methods exploit the non-uniformity of OPD signals by pr…
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
Yuanyi Wang, Yifan Yang, Su Lu +9
Continual post-training aims to extend large language models (LLMs) with new knowledge, skills, and behaviors, yet it remains unclear when sequential updates enable capability tran…
Expandable Residual Approximation for Knowledge Distillation
Zhaoyi Yan, Binghui Chen, Yunfan Liu +1
Knowledge distillation (KD) aims to transfer knowledge from a large-scale teacher model to a lightweight one, significantly reducing computational and storage requirements. However…
Delving Deep into Semantic Relation Distillation
Zhaoyi Yan, Kangjun Liu, Qixiang Ye
Knowledge distillation has become a cornerstone technique in deep learning, facilitating the transfer of knowledge from complex models to lightweight counterparts. Traditional dist…
Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting
Mingyue Guo, Binghui Chen, Zhaoyi Yan +2
Multidomain crowd counting aims to learn a general model for multiple diverse datasets. However, deep networks prefer modeling distributions of the dominant domains instead of all…
Regressor-Segmenter Mutual Prompt Learning for Crowd Counting
Mingyue Guo, Li Yuan, Zhaoyi Yan +3
Crowd counting has achieved significant progress by training regressors to predict instance positions. In heavily crowded scenarios, however, regressors are challenged by uncontrol…