TOBYQA: A Time-Augmented Model-Based Method for Derivative-Free Optimization under Noise and Temporal Drift

Abstract

Derivative-free optimization (DFO) is challenging when the observation channel varies over time and evaluations are noisy. Conventional model-based methods assume stationary observations; under temporal drift, historical data biases gradient estimates and acceptance tests confound latent-objective decrease with temporal variation. We propose TOBYQA (Time-augmented Optimization BY Quadratic Approximation), a regularized model-based DFO framework that jointly incorporates spatial geometry and temporal variations within a single saddle-point interpolation system. TOBYQA augments the classical least-Frobenius-norm quadratic interpolation system with a linear-in-time drift term and a ridge regularization on the residual kernel, which accommodates noise and ensures well-posedness when the constraint block has full column rank, relaxing the geometric poisedness requirements of classical interpolation. We prove that under affine temporal drift, the recovered gradient is algebraically invariant to the drift rate for any noise scale and sample radius. This property leads to a drift-compensated acceptance test that subtracts the estimated temporal component from the observed reduction. Driven by an adaptive cubic regularization scheme with a closed-form step and a geometry-guarded statistical stationarity stopping rule, TOBYQA achieves an expected oracle complexity of $O({\epsilon }^{-2})$. Benchmark evaluations across diverse temporal drift regimes show that, at tolerance $\tau =10^{-3}$, TOBYQA solves 71.0%, 60.8%, and 41.9% of instances at $n=6$, $10$, and $20$, respectively, compared with 25.1%, 18.5%, and 14.9% for the best-performing comparison method. These results demonstrate higher solve rates under temporal drift while retaining comparable performance in static environments.

22 pages, 2 figures, 6 tables