On the Role of Initialization on the Implicit Bias in Deep Linear Networks

Oria Gruber, Haim Avron

Despite Deep Learning's (DL) empirical success, our theoretical understanding of its efficacy remains limited. One notable paradox is that while conventional wisdom discourages per…