Optimal Parameter-Free First-Order Methods for Convex Optimization with Unknown Growth and Smoothness
arXiv:2607.11878
The paper introduces parameter-free first-order optimization methods that automatically adapt to unknown smoothness and growth properties of convex functions, achieving optimal convergence rates across various regimes.
Abstract
We study deterministic first-order minimization of a convex function without prior knowledge of the objective's growth, smoothness regime, or associated parameters. We develop anytime, parameter-free bundle-level methods that adapt simultaneously to these unknown properties and attain best-known oracle complexities. For nonsmooth Lipschitz objectives satisfying quadratic growth, the proposed bundle-level W-certificate method (BLW) achieves the optimal complexity without requiring the growth modulus or target accuracy as input. We then introduce an accelerated variant, A-BLW. Without knowing the Hölder smoothness parameters, the quadratic-growth modulus, or the target accuracy, A-BLW attains the optimal rates in the nonsmooth, weakly smooth, and smooth regimes. Central to both methods is an affine W-certificate, a condition based on the descent-slowness of an affine minorant that converts the geometry of a bundle model into an optimality-gap guarantee under quadratic growth. A stopping-time analysis further shows that the same A-BLW algorithm, without modification, achieves the corresponding best-known rates for general convex objectives and for objectives satisfying Hölder growth of order at least two. Numerical experiments illustrate the practical performance of the proposed methods.
39 pages, 4 figures, 1 table