57 citations · 148 across the 15 of their papers we have counts for
17 papers
On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers
Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5
We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles…
Exploiting Structure with Anisotropic Consensus-Based Optimization
Sabrina Bonandin, Konstantin Riedl, Sara Veneruso
Anisotropic consensus-based optimization (CBO), a multi-agent metaheuristic derivative-free optimization method, which reliably finds global minima of nonsmooth and nonconvex objec…
Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit
Konstantin Riedl, Konstantinos Spiliopoulos, Justin Sirignano
A convergence analysis is developed for the regularized Newton method for training neural networks (NNs) in the overparameterized limit. As the number of hidden units tends to infi…
Convergence of Consensus-Based Particle Methods for Nonconvex Bi-Level Optimization
Yutong Chao, Xudong Sun, Konstantin Riedl +2
In this paper, we study a consensus-based optimization method for nonconvex bi-level optimization, where the objective is to minimize an upper-level function over the set of global…
Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
Albert Alcalde, Leon Bungert, Konstantin Riedl +1
Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the e…
Strong Global Convergence of the Consensus-Based Optimization Algorithm
Sabrina Bonandin, Konstantin Riedl, Sara Veneruso
Consensus-based optimization (CBO) is a multi-agent metaheuristic derivative-free optimization algorithm that has proven to be capable of globally minimizing nonconvex nonsmooth fu…
Global Convergence of Adjoint-Optimized Neural PDEs
Konstantin Riedl, Justin Sirignano, Konstantinos Spiliopoulos
Many engineering and scientific fields have recently become interested in modeling terms in partial differential equations (PDEs) with neural networks, which requires solving the i…
Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
Nicolás García Trillos, Aditya Kumar Akash, Sixu Li +2
Adversarial attacks pose significant challenges in many machine learning applications, particularly in the setting of distributed training and federated learning, where malicious a…
CBO: Consensus-Based Bi-Level Optimization
Nicolás García Trillos, Sixu Li, Konstantin Riedl +1
Bi-level optimization problems, where one wishes to find the global minimizer of an upper-level objective function over the globally optimal solution set of a lower-level objective…
CBX: Python and Julia packages for consensus-based interacting particle methods
Rafael Bailo, Alethea Barbaro, Susana N. Gomes +4
We introduce CBXPy and ConsensusBasedX.jl, Python and Julia implementations of consensus-based interacting particle systems (CBX), which generalise consensus-based optimization met…
Consensus-Based Optimization with Truncated Noise
Massimo Fornasier, Peter Richtárik, Konstantin Riedl +1
Consensus-based optimization (CBO) is a versatile multi-particle metaheuristic optimization method suitable for performing nonconvex and nonsmooth global optimizations in high dime…
Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent
Konstantin Riedl, Timo Klock, Carina Geldhauser +1
In this paper, we provide a novel analytical perspective on the theoretical understanding of gradient-based learning algorithms by interpreting consensus-based optimization (CBO),…
Consensus-Based Optimization for Saddle Point Problems
Hui Huang, Jinniao Qiu, Konstantin Riedl
In this paper, we propose consensus-based optimization for saddle point problems (CBO-SP), a novel multi-particle metaheuristic derivative-free optimization method capable of prova…
Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law
Konstantin Riedl
In this paper we study consensus-based optimization (CBO), a versatile, flexible and customizable optimization method suitable for performing nonconvex and nonsmooth global optimiz…
On the Global Convergence of Particle Swarm Optimization Methods
Hui Huang, Jinniao Qiu, Konstantin Riedl
In this paper we provide a rigorous convergence analysis for the renowned particle swarm optimization method by using tools from stochastic calculus and the analysis of partial dif…
Convergence of Anisotropic Consensus-Based Optimization in Mean-Field Law
Massimo Fornasier, Timo Klock, Konstantin Riedl
In this paper we study anisotropic consensus-based optimization (CBO), a multi-agent metaheuristic derivative-free optimization method capable of globally minimizing nonconvex and…
Consensus-Based Optimization Methods Converge Globally
Massimo Fornasier, Timo Klock, Konstantin Riedl
In this paper, we study consensus-based optimization (CBO), which is a multi-agent metaheuristic derivative-free optimization method that can globally minimize nonconvex nonsmooth…