4 papers
Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer
Jiahe Fan, Si Chen, Yinghao Hou +4
Specialized models encode task-oriented behavior, but transferring that behavior to a general language model usually requires training, distillation, or representation alignment. W…
Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach
Yinghao Hou, Jiahe Fan, Yuanhao Pu +2
Training-free fusion of heterogeneous multimodal large language models (MLLMs) provides a direct route for cross-scale capability transfer, yet improvements in aggregate performanc…
Training-Free Knowledge Transfer Across Model Scales through Activation-Guided Pruning
Jiahe Fan, Si Chen, Yinghao Hou +2
Heterogeneous model fusion seeks to combine models that differ in tasks, initializations, architectures, or scales. We study an underexplored cross-scale setting: improving a small…
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
Jiahe Fan, Yinghao Hou, Si Chen +3
Can large language models with substantially different parameter spaces be merged by direct weighted averaging, without training or semantic alignment? Existing heterogeneous fusio…