InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models

Yanggan Gu, Yuanyi Wang, Zhaoyi Yan +4

Model fusion combines multiple Large Language Models (LLMs) with different strengths into a more powerful, integrated model through lightweight training methods. Existing works on…