A Spectral Condition for Feature Learning

Greg Yang, James B. Simon, Jeremy Bernstein

The push to train ever larger neural networks has motivated the study of initialization and training at large network width. A key challenge is to scale training so that a network'…