24 citations · 87 across the 20 of their papers we have counts for
20 papers
Consensus-based optimization for linearly separable functions
Christian Fiedler, Tim Roith
Consensus-based optimization (CBO) is an efficient metaheuristic for global optimisation with attractive mathematical properties, allowing global convergence results even in non-co…
Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers
Ahmad Aloradi, Tim Roith, Emanuël A. P. Habets +1
Sparse training reduces the memory and computational costs of deep neural networks. However, sparse optimization methods, e.g., those adding an penalty, often control spar…
Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
Albert Alcalde, Leon Bungert, Konstantin Riedl +1
Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the e…
Allure of Craquelure: A Variational-Generative Approach to Crack Detection in Paintings
Laura Paul, Holger Rauhut, Martin Burger +2
Recent advances in imaging technologies, deep learning and numerical performance have enabled non-invasive detailed analysis of artworks, supporting their documentation and conserv…
Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference
Simon Welker, Lorenz Kuger, Tim Roith +4
In this work, we present and investigate the novel blind inverse problem of position-blind ptychography, i.e., ptychographic phase retrieval without any knowledge of scan positions…
Introduction to Regularization and Learning Methods for Inverse Problems
Danielle Bednarski, Tim Roith
These lecture notes evolve around mathematical concepts arising in inverse problems. We start by introducing inverse problems through examples such as differentiation, deconvolutio…
Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies
Tim Roith, Leon Bungert, Philipp Wacker
Consensus-based optimization (CBO) has established itself as an efficient gradient-free optimization scheme, with attractive mathematical properties, such as mean-field convergence…
MirrorCBO: A consensus-based optimization method in the spirit of mirror descent
Leon Bungert, Franca Hoffmann, Dohyeon Kim +1
In this work we propose MirrorCBO, a consensus-based optimization (CBO) method which generalizes standard CBO in the same way that mirror descent generalizes gradient descent. For…
Analysis of mean-field models arising from self-attention dynamics in transformer architectures with layer normalization
Martin Burger, Samira Kabri, Yury Korolev +2
The aim of this paper is to provide a mathematical analysis of transformer architectures using a self-attention mechanism with layer normalization. In particular, observed patterns…
Adversarial flows: A gradient flow characterization of adversarial attacks
Lukas Weigand, Tim Roith, Martin Burger
A popular method to perform adversarial attacks on neuronal networks is the so-called fast gradient sign method and its iterative variant. In this paper, we interpret this method a…
CBX: Python and Julia packages for consensus-based interacting particle methods
Rafael Bailo, Alethea Barbaro, Susana N. Gomes +4
We introduce CBXPy and ConsensusBasedX.jl, Python and Julia implementations of consensus-based interacting particle systems (CBX), which generalise consensus-based optimization met…
Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flow
Tjeerd Jan Heeringa, Tim Roith, Christoph Brune +1
This paper presents a method for finding a sparse representation of Barron functions. Specifically, given an function , the inverse scale space flow is used to find a spar…
Resolution-Invariant Image Classification based on Fourier Neural Operators
Samira Kabri, Tim Roith, Daniel Tenbrinck +1
In this paper we investigate the use of Fourier Neural Operators (FNOs) for image classification in comparison to standard Convolutional Neural Networks (CNNs). Neural operators ar…
Polarized consensus-based dynamics for optimization and sampling
Leon Bungert, Tim Roith, Philipp Wacker
In this paper we propose polarized consensus-based dynamics in order to make consensus-based optimization (CBO) and sampling (CBS) applicable for objective functions with several g…
Ratio convergence rates for Euclidean first-passage percolation: Applications to the graph infinity Laplacian
Leon Bungert, Jeff Calder, Tim Roith
In this paper we prove the first quantitative convergence rates for the graph infinity Laplace equation for length scales at the connectivity threshold. In the graph-based semi-sup…
Uniform Convergence Rates for Lipschitz Learning on Graphs
Leon Bungert, Jeff Calder, Tim Roith
Lipschitz learning is a graph-based semi-supervised learning method where one extends labels from a labeled to an unlabeled data set by solving the infinity Laplace equation on a w…
Neural Architecture Search via Bregman Iterations
Leon Bungert, Tim Roith, Daniel Tenbrinck +1
We propose a novel strategy for Neural Architecture Search (NAS) based on Bregman iterations. Starting from a sparse neural network our gradient-based one-shot algorithm gradually…
A Bregman Learning Framework for Sparse Neural Networks
Leon Bungert, Tim Roith, Daniel Tenbrinck +1
We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We deriv…
CLIP: Cheap Lipschitz Training of Neural Networks
Leon Bungert, René Raab, Tim Roith +2
Despite the large success of deep neural networks (DNN) in recent years, most neural networks still lack mathematical guarantees in terms of stability. For instance, DNNs are vulne…
Continuum Limit of Lipschitz Learning on Graphs
Tim Roith, Leon Bungert
Tackling semi-supervised learning problems with graph-based methods has become a trend in recent years since graphs can represent all kinds of data and provide a suitable framework…