14 papers
Sparse entropic quadrature for moment equations
Georgii Oblapenko, Michael Herty, Manuel Torrilhon
In the present work, we study the properties of the sparse entropic quadrature method, a recently proposed method for the closure of systems of moment equations in rarefied gas dyn…
A Mean-Field Theory of Transformers: Well-Posedness of the Coupled Data--Parameter Dynamics and Global Convergence of Training
Michael Herty, Hailiang Liu
We develop a rigorous mean-field theory for transformer networks that captures two large-scale limits inherent in the architecture: the number of tokens in the input s…
Convergence of time-discrete finite particle consensus based optimization in Hilbert spaces
Michael Herty, Hui Huang, Hicham Kouhkouh
We study a time-discrete, finite-particle Consensus-Based Optimization (CBO) algorithm in a separable Hilbert space. Our analysis provides convergence guarantees directly for the c…
A PDE-Based Framework for Generative Modeling Beyond Classical Score-Based Diffusion
Horacio Tettamanti, Michael Herty
We introduce an alternative generative framework based on a nonlinear modification of the classical Ornstein--Uhlenbeck dynamics. The proposed dynamics admits both a microscopic de…
Ensemble Kalman Inversion as an Inertial Interacting Particle System
Michael Herty, Pierpaolo Porretta, Giuseppe Visconti
Ensemble Kalman Inversion (EKI) is a derivative-free, ensemble-based method for inverse and optimization problems. A central limitation of its continuous-time dynamics is premature…
Sparse and low-rank kinetic distribution estimation
Georgii Oblapenko, Lambert Theisen, Rostislav-Paul Wilhelm +2
In this paper, we consider methods that allow for memory-efficient storage of high-dimensional distributions and retain certain key features thereof, specifically in a kinetic theo…
Computation of entanglement for quantum states by a Consensus-Based Optimization method
Michael Herty, Yijia Tang, Yizhou Zhou
The computation of quantum entanglement can be formulated as a high-dimensional nonconvex optimization problem with orthogonality constraints. In this work, we propose structure-pr…
Consensus-based optimization with -stable jump processes
Pedro Aceves-Sanchez, Giacomo Albi, Federica Ferrarese +1
In this paper, we introduce a CBO method that incorporates jumps driven by an -stable stochastic process within a kinetic framework. This approach naturally induces nonlocal sto…
Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization
William De Deyn, Michael Herty, Giovanni Samaey
We study Consensus-Based Optimization (CBO) for two-layer neural network training. We compare the performance of CBO against Adam on two test cases and demonstrate how a hybrid app…
Sparse Reconstruction of Multi-Dimensional Kinetic Distributions
Georgii Oblapenko, Manuel Torrilhon, Michael Herty
In the present work, we propose a novel method for reconstruction of multi-dimensional kinetic distributions, based on their representation as a mixture of Dirac delta functions. T…
Kinetic simulated annealing optimization with entropy-based cooling rate
Michael Herty, Mattia Zanella
We present a modified simulated annealing method with a dynamical choice of the cooling temperature. The latter is determined via a closed-loop control and is proven to yield expon…
Kinetic variable-sample methods for stochastic optimization problems
Sabrina Bonandin, Michael Herty
We discuss kinetic-based particle optimization methods and variable-sample strategies for problems where the cost function represents the expected value of a random mapping. Kineti…
Consensus-based algorithms for stochastic optimization problems
Sabrina Bonandin, Michael Herty
We address an optimization problem where the cost function is the expectation of a random mapping. To tackle the problem two approaches based on the approximation of the objective…
Relaxation and asymptotic expansion of controlled stiff differential equations
Michael Herty, Hicham Kouhkouh
The control of relaxation-type systems of ordinary differential equations is investigated using the Hamilton-Jacobi-Bellman equation. First, we recast the model as a singularly per…