Statistically Optimal First Order Algorithms: A Proof via Orthogonalization

Andrea Montanari, Yuchen Wu

We consider a class of statistical estimation problems in which we are given a random data matrix $\mathbf{X}\in {\mathbb{R}}^{n\times d}$ (and possibly some labels ${\boldsymb…