The Iterates of the Frank-Wolfe Algorithm May Not Converge

Abstract

The Frank-Wolfe algorithm is a popular method for minimizing a smooth convex function $f$ over a compact convex set $\mathcal{C}$. While many convergence results have been derived in terms of function values, hardly nothing is known about the convergence behavior of the sequence of iterates $(x_t{)}_{t\in \mathbb{N}}$. Under the usual assumptions, we design several counterexamples to the convergence of $(x_t{)}_{t\in \mathbb{N}}$, where $f$ is $d$-time continuously differentiable, $d\ge 2$, and $f(x_t)\to {\min }_{\mathcal{C}}f$. Our counterexamples cover the cases of open-loop, closed-loop, and line-search step-size strategies. We do not assume \emph{misspecification} of the linear minimization oracle and our results thus hold regardless of the points it returns, demonstrating the fundamental pathologies in the convergence behavior of $(x_t{)}_{t\in \mathbb{N}}$.

15 pages, 7 figures