Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Can Model Merging Improve Aggregation in DiLoCo?
Stefan Horoi, Benjamin Thérien, Guy Wolf +1
Model merging techniques, which aggregate independently finetuned models into one to combine their capabilities, have become a topic of significant interest in recent years, with a…
cs.LG2026
From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging
Stefan Horoi, Guy Wolf, Eugene Belilovsky +1
Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized datasets. This has led to a proliferation of ex…