The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold

Jialin Mao, Itay Griniasty, Han Kheng Teoh +5

We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilis…