Informative Fisher Modes in Neural Networks as a Result of Explicit Symmetry Breaking

Ibrahim Talha Ersoy, Björn Ladewig, Karoline Wiesner

Learning in a neural network takes place on a high dimensional, non-convex loss landscape, the Hessian curvature of which dictates the learning dynamics. Despite its high dimension…