4 papers
Model Assembly Learning with Heterogeneous Layer Weight Merging
Yi-Kai Zhang, Jin Wang, Xu-Xiang Zhong +2
Model merging acquires general capabilities without extra data or training by combining multiple models' parameters. Previous approaches achieve linear mode connectivity by alignin…
Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
Yi-Kai Zhang, De-Chuan Zhan, Han-Jia Ye
Large Language Models (LLMs) have demonstrated human-like instruction-following abilities, particularly those exceeding 100 billion parameters. The combined capability of some smal…
OmniEvalKit: A Modular, Lightweight Toolbox for Evaluating Large Language Model and its Omni-Extensions
Yi-Kai Zhang, Xu-Xiang Zhong, Shiyin Lu +3
The rapid advancements in Large Language Models (LLMs) have significantly expanded their applications, ranging from multilingual support to domain-specific tasks and multimodal int…
Wings: Learning Multimodal LLMs without Text-only Forgetting
Yi-Kai Zhang, Shiyin Lu, Yang Li +7
Multimodal large language models (MLLMs), initiated with a trained LLM, first align images with text and then fine-tune on multimodal mixed inputs. However, the MLLM catastrophical…