4 papers · 1 filter
InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion
Yuanyi Wang, Zhaoyi Yan, Yiming Zhang +4
Recent advances in large language models (LLMs) have intensified efforts to fuse heterogeneous open-source models into a unified system that inherits their complementary strengths.…
InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
Congkai Xie, Shuo Cai, Wenjun Wang +17
Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as…
InfiFusion: A Unified Framework for Enhanced Cross-Model Reasoning via LLM Fusion
Zhaoyi Yan, Yiming Zhang, Baoyi He +7
We introduce InfiFusion, an efficient training pipeline designed to integrate multiple domain-specialized Large Language Models (LLMs) into a single pivot model, effectively harnes…
Unconstrained Model Merging for Enhanced LLM Reasoning
Yiming Zhang, Baoyi He, Shengyu Zhang +12
Recent advancements in building domain-specific large language models (LLMs) have shown remarkable success, especially in tasks requiring reasoning abilities like logical inference…