An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models

Jialin Mao, Itay Griniasty, Yan Sun +3

Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regul…