Binary interaction methods for high dimensional global optimization and machine learning

Abstract

In this work we introduce a new class of gradient-free global optimization methods based on a binary interaction dynamics governed by a Boltzmann type equation. In each interaction the particles act taking into account both the best microscopic binary position and the best macroscopic collective position. In the mean-field limit we show that the resulting Fokker-Planck partial differential equations generalize the current class of consensus based optimization (CBO) methods. For the latter methods, convergence to the global minimizer can be shown for a large class of functions. Algorithmic implementations inspired by the well-known direct simulation Monte Carlo methods in kinetic theory are derived and discussed. Several examples on prototype test functions for global optimization are reported including applications to machine learning.

References in corpus (5)

• A Literature Survey of Benchmark Functions For Global Optimization Problems
• A consensus-based model for global optimization and its mean-field limit
• On the Random Batch Method for second order interacting particle systems
• Consensus-Based Optimization on Hypersurfaces: Well-Posedness and Mean-Field Limit
• A consensus-based global optimization method with adaptive momentum estimation

Cited by in corpus (6)

• On the mean-field limit for the consensus-based optimization
• Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning
• Consensus based optimization via jump-diffusion stochastic differential equations
• Consensus based optimization with memory effects: random selection and applications
• Swarm-based optimization with jumps: a kinetic BGK framework and convergence analysis
• Kinetic variable-sample methods for stochastic optimization problems