Global optimization of Lipschitz functions

Abstract

The goal of the paper is to design sequential strategies which lead to efficient optimization of an unknown function under the only assumption that it has a finite Lipschitz constant. We first identify sufficient conditions for the consistency of generic sequential algorithms and formulate the expected minimax rate for their performance. We introduce and analyze a first algorithm called LIPO which assumes the Lipschitz constant to be known. Consistency, minimax rates for LIPO are proved, as well as fast rates under an additional Hölder like condition. An adaptive version of LIPO is also introduced for the more realistic setup where the Lipschitz constant is unknown and has to be estimated along with the optimization. Similar theoretical guarantees are shown to hold for the adaptive LIPO algorithm and a numerical assessment is provided at the end of the paper to illustrate the potential of this strategy with respect to state-of-the-art methods over typical benchmark problems for global optimization.

References in corpus (5)

• A Literature Survey of Benchmark Functions For Global Optimization Problems
• BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
• Stochastic simultaneous optimistic optimization
• Black-box optimization of noisy functions with unknown smoothness
• A ranking approach to global optimization

Cited by in corpus (7)

• A Systematic Approach to Kähler Moduli Stabilisation
• A Framework of Transfer Learning in Object Detection for Embedded Systems
• On the separation of correlation-assisted sum capacities of multiple access channels
• Deep Out-of-Distribution Uncertainty Quantification via Weight Entropy Maximization
• BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning
• Predictive Local Smoothness for Stochastic Gradient Methods
• Inverse Graphical Method for Global Optimization and Application to Design Centering Problem