Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse
Ekansh Sharma, Daniel M. Roy, Gintare Karolina Dziugaite
Model merging aims to efficiently combine the weights of multiple expert models, each trained on a specific task, into a single multi-task model, with strong performance across all…
cs.LG2024
Simultaneous linear connectivity of neural networks modulo permutation
Ekansh Sharma, Devin Kwok, Tom Denton +3
Neural networks typically exhibit permutation symmetries which contribute to the non-convexity of the networks' loss landscapes, since linearly interpolating between two permuted v…