1 citations · 2 across the 14 of their papers we have counts for
6 papers · 1 filter
Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation
Jingang Zhou, Yuyi Zhou, Haiyang Guo +6
On-policy distillation (OPD) is a promising approach for transferring knowledge between language models, where a student receives dense token-level supervision along its own genera…
CAMFT: Conflict-Aware Mergeable Fine-Tuning for Large Language Models
Jingang Zhou, Haiyang Guo, Yuan Ma +2
Model merging has emerged as a promising paradigm for integrating multiple task-specific capabilities into a single large language model. However, existing methods predominantly fo…
Denser Better: Limits of On-Policy Self-Distillation for Continual Post-Training
Meng Wang, Haohan Zhao, Wenzhuo Liu +7
Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgett…
Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
Haiyang Guo, Fanhu Zeng, Fei Zhu +9
The rapid advancement of generative models has empowered modern AI systems to comprehend and produce highly sophisticated content, even achieving human-level performance in specifi…
Federated Continual Instruction Tuning
Haiyang Guo, Fanhu Zeng, Fei Zhu +5
A vast amount of instruction tuning data is crucial for the impressive performance of Large Multimodal Models (LMMs), but the associated computational costs and data collection dem…
DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning
Haiyang Guo, Fei Zhu, Fanhu Zeng +2
Continual learning aims to equip models with the ability to retain previously learned knowledge like a human. Recent work incorporating Parameter-Efficient Fine-Tuning has revitali…