Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer

Jiahe Fan, Si Chen, Yinghao Hou +4

Specialized models encode task-oriented behavior, but transferring that behavior to a general language model usually requires training, distillation, or representation alignment. W…