Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe

Abstract

Stochastic compositional optimization minimizes $F(f(x),x)$ over a compact convex set when the inner map $f$ is available only through noisy samples. We propose Hybrid Momentum Stochastic Frank--Wolfe, a projection-free method for outer functions $F$ that are jointly convex and Lipschitz but not necessarily differentiable, such as maxima, CVaR, and norms. The method tracks the Jacobian and the value of $f$ with momentum, using a Taylor correction for the value, and passes the tracked linearization to a generalized linear minimization oracle. Under finite-variance noise, it attains an expected generalized Frank--Wolfe gap of order $K^{-1/4}$ for non-convex objectives and, when the linearized models minorize the objective, a last-iterate primal error of order $K^{-1/3}$. The analysis extends to heavy-tailed noise without clipping, with separate moment orders for values and Jacobians. With an exact oracle, the method reduces to the deterministic method of Vladarean et al. (2023).