Swarm-based optimization with jumps: a kinetic BGK framework and convergence analysis

Abstract

Metaheuristic algorithms are powerful tools for global optimization, particularly for non-convex and non-differentiable problems where exact methods are often impractical. Particle-based optimization methods, inspired by swarm intelligence principles, have shown effectiveness due to their ability to balance exploration and exploitation within the search space. In this work, we introduce a novel particle-based optimization algorithm where velocities are updated via random jumps, a strategy commonly used to enhance stochastic exploration. We formalize this approach by describing the dynamics through a kinetic modelling of BGK type, offering a unified framework that accommodates general noise distributions, including heavy-tailed ones like Cauchy. Under suitable parameter scaling, the model reduces to the Consensus-Based Optimization (CBO) dynamics. For non-degenerate Gaussian noise in bounded domains, we prove propagation of chaos and convergence towards minimizers. Numerical results on benchmark problems validate the approach and highlight its connection to CBO.

References in corpus (14)

• A Literature Survey of Benchmark Functions For Global Optimization Problems
• A consensus-based model for global optimization and its mean-field limit
• Propagation of chaos: a review of models, methods and applications. II. Applications
• On the Global Convergence of Particle Swarm Optimization Methods
• Consensus-Based Optimization on Hypersurfaces: Well-Posedness and Mean-Field Limit
• On the mean-field limit for the consensus-based optimization
• Consensus-Based Optimization Methods Converge Globally
• Consensus based optimization via jump-diffusion stochastic differential equations
• Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law
• Binary interaction methods for high dimensional global optimization and machine learning
• Mean-field limits for Consensus-Based Optimization and Sampling
• Consensus based optimization with memory effects: random selection and applications
• Kinetic description and convergence analysis of genetic algorithms for global optimization
• A stochastic particle system approximating the BGK equation