Riemannian Gradient Descent for Low-Rank Architectures

Nicholas Knight

We explore Riemannian optimization techniques for rank-factored matrix parameters, targeting contemporary deep learning applications. We examine ten points in the algorithm design…