An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scale

Adrián Javaloy, Antonio Vergari

Orthogonality constraints are ubiquitous in robust and probabilistic machine learning. Unfortunately, current optimizers are computationally expensive and do not scale to problems…