A consensus-based model for global optimization and its mean-field limit

Abstract

We introduce a novel first-order stochastic swarm intelligence (SI) model in the spirit of consensus formation models, namely a consensus-based optimization (CBO) algorithm, which may be used for the global optimization of a function in multiple dimensions. The CBO algorithm allows for passage to the mean-field limit, which results in a nonstandard, nonlocal, degenerate parabolic partial differential equation (PDE). Exploiting tools from PDE analysis we provide convergence results that help to understand the asymptotic behavior of the SI model. We further present numerical investigations underlining the feasibility of our approach.

References in corpus (3)

• Novel type of phase transition in a system of self-driven particles
• A Literature Survey of Benchmark Functions For Global Optimization Problems
• First--order continuous models of opinion formation

Cited by in corpus (52)

• Neural Network Approximation: Three Hidden Layers Are Enough
• Propagation of chaos: a review of models, methods and applications. II. Applications
• Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
• On the Global Convergence of Particle Swarm Optimization Methods
• Consensus-Based Optimization on Hypersurfaces: Well-Posedness and Mean-Field Limit
• Instantaneous control of interacting particle systems in the mean-field limit
• Mean Field Limits for Interacting Diffusions in a Two-Scale Potential
• On the mean-field limit for the consensus-based optimization
• Consensus-Based Optimization Methods Converge Globally
• Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning
• Consensus based optimization via jump-diffusion stochastic differential equations
• Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law
• Binary interaction methods for high dimensional global optimization and machine learning
• Controlled pattern formation of stochastic Cucker-Smale systems with network structures
• Consensus-Based Optimization for Saddle Point Problems
• Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law
• Dynamics of the Desai-Zwanzig model in multi-well and random energy landscapes
• A consensus-based global optimization method for high dimensional machine learning problems
• Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics
• Polarized consensus-based dynamics for optimization and sampling
• Consensus based optimization with memory effects: random selection and applications
• Propagation of chaos: a review of models, methods and applications. I. Models and methods
• Mean-field limits for Consensus-Based Optimization and Sampling
• On the Generalised Langevin Equation for Simulated Annealing
• CBX: Python and Julia packages for consensus-based interacting particle methods
• Mean-field optimal control for biological pattern formation
• Self-interacting CBO: Existence, uniqueness, and long-time convergence
• Convergence of a first-order consensus-based global optimization algorithm
• Consensus-Based Optimization with Truncated Noise
• A multiscale Consensus-Based algorithm for multi-level optimization
• Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
• Well-posedness and approximation of reflected McKean-Vlasov SDEs with applications
• Body-attitude coordination in arbitrary dimension
• The sharp, the flat and the shallow: Can weakly interacting agents learn to escape bad minima?
• Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
• MirrorCBO: A consensus-based optimization method in the spirit of mirror descent
• PDE-Constrained Optimization Models and Pseudospectral Methods for Multiscale Particle Dynamics
• Convergence and error estimates for time-discrete consensus-based optimization algorithms
• PDE-constrained optimal control of a leader-follower opinion formation model
• Consensus-based qubit configuration optimization for variational algorithms on neutral atom quantum systems
• Uniform-in-time propagation of chaos for the Cucker--Smale model
• Consensus-based algorithms for stochastic optimization problems
• Adversarial flows: A gradient flow characterization of adversarial attacks
• EnKSGD: A Class Of Preconditioned Black Box Optimization And Inversion Algorithms
• On the exponential ergodicity of the McKean-Vlasov SDE depending on a polynomial interaction
• One dimensional consensus based algorithm for non-convex optimization
• A Constrained Consensus Based Optimization algorithm and its Application to Finance
• Optimal control for interacting particle systems driven by neural networks
• Mean-field limits for interacting diffusions with colored noise: phase transitions and spectral numerical methods
• Swarm-based optimization with jumps: a kinetic BGK framework and convergence analysis
• Well-posedness and propagation of chaos for multi-agent models with strategies and diffusive effects
• Micro-Macro Decomposition of Particle Swarm Optimization Methods