Beyond Polytopes: Damped Newton Frank--Wolfe Methods over Compact Convex Sets

Abstract

We study convex optimization over compact convex sets for twice differentiable objective functions with positive definite Hessians, assuming access to the feasible set through a linear minimization oracle. Existing second-order Frank--Wolfe methods either provide only a local linear rate over general convex sets or achieve global linear and local quadratic convergence only over polytopes. We propose damped Newton Frank--Wolfe methods that approximately solve constrained damped Newton subproblems using Frank--Wolfe or away-step Frank--Wolfe inner iterations. We develop residual backtracking, using a root Newton stepsize as a safe reference, together with a switching rule that eventually takes full steps. Under Hölder smoothness assumptions for $p$th derivatives, we establish global linear convergence and local convergence with Q-order at least $1+\nu$ ($p=2$) and local quadratic convergence ($p=3$). Experiments on matrix sensing and ridge-regularized logistic regression show that the residual-backtracking variant is consistently robust and often achieves the best performance among the tested first- and second-order baselines.

arXiv comment37 pages, 11 figures