Analyzing the Role of Permutation Invariance in Linear Mode Connectivity

Keyao Zhan, Puheng Li, Lei Wu

It was empirically observed in Entezari et al. (2021) that when accounting for the permutation invariance of neural networks, there is likely no loss barrier along the linear inter…