Simultaneous linear connectivity of neural networks modulo permutation

Ekansh Sharma, Devin Kwok, Tom Denton +3

Neural networks typically exhibit permutation symmetries which contribute to the non-convexity of the networks' loss landscapes, since linearly interpolating between two permuted v…