3 papers
cs.LG2026
Can Model Merging Improve Aggregation in DiLoCo?
Stefan Horoi, Benjamin Thérien, Guy Wolf +1
Model merging techniques, which aggregate independently finetuned models into one to combine their capabilities, have become a topic of significant interest in recent years, with a…
cs.CL2025
Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs
Stefan Horoi, Sangwoo Cho, Supriyo Chakraborty +4
Task arithmetic is a powerful technique for transferring skills between Large Language Models (LLMs), but it often suffers from negative interference when models have diverged duri…
cs.LG2024
Non-Uniform Parameter-Wise Model Merging
Albert Manuel Orozco Camacho, Stefan Horoi, Guy Wolf +1
Combining multiple machine learning models has long been a technique for enhancing performance, particularly in distributed settings. Traditional approaches, such as model ensemble…