4 papers
Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
Albert Alcalde, Leon Bungert, Konstantin Riedl +1
Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the e…
Duality for the Adversarial Total Variation
Leon Bungert, Lucas Schmitt
Adversarial training of binary classifiers can be reformulated as regularized risk minimization involving a nonlocal total variation. Building on this perspective, we establish a c…
MirrorCBO: A consensus-based optimization method in the spirit of mirror descent
Leon Bungert, Franca Hoffmann, Dohyeon Kim +1
In this work we propose MirrorCBO, a consensus-based optimization (CBO) method which generalizes standard CBO in the same way that mirror descent generalizes gradient descent. For…
Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies
Tim Roith, Leon Bungert, Philipp Wacker
Consensus-based optimization (CBO) has established itself as an efficient gradient-free optimization scheme, with attractive mathematical properties, such as mean-field convergence…