Architecture Induces Structural Invariant Manifolds of Neural Network Training Dynamics

Jiajie Zhao, Tao Luo, Yaoyu Zhang

While architecture is recognized as key to the performance of deep neural networks, its precise effect on training dynamics has been unclear due to the confounding influence of dat…