Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning

Daniel Kunin, Allan Raventós, Clémentine Dominé +4

While the impressive performance of modern neural networks is often attributed to their capacity to efficiently extract task-relevant features from data, the mechanisms underlying…