Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence
arXiv:2006.01719
Abstract
We develop a novel variant of the classical Frank-Wolfe algorithm, which we call spectral Frank-Wolfe, for convex optimization over a spectrahedron. The spectral Frank-Wolfe algorithm has a novel ingredient: it computes a few eigenvectors of the gradient and solves a small-scale SDP in each iteration. Such procedure overcomes slow convergence of the classical Frank-Wolfe algorithm due to ignoring eigenvalue coalescence. We demonstrate that strict complementarity of the optimization problem is key to proving linear convergence of various algorithms, such as the spectral Frank-Wolfe algorithm as well as the projected gradient method and its accelerated version.
The main text has nine pages. Reading the first three sections should give a good understanding of the paper and should take around 15 minutes. Published in proceedings of the 37th International Conference on Machine Learning, online, PMLR 119, 2020