PCA of high dimensional random walks with comparison to neural network training

Joseph M. Antognini, Jascha Sohl-Dickstein

One technique to visualize the training of neural networks is to perform PCA on the parameters over the course of training and to project to the subspace spanned by the first few P…