7 papers
Quasi Linear Kernel Attention with Infinite Capacity
Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer
The evaluation cost of transformers with softmax attention scales quadratically with sequence length. Kernel attention addresses this by replacing softmax with a more general kerne…
Fast Gauss Sums via Flash Attention
Nicolaj Rux, Sebastian Neumayer
Gaussian kernel sums are the computational core of maximum mean discrepancies (MMDs), kernel gradient flows, Stein variational gradient descent (SVGD), and many other kernel method…
A Stability Benchmark of Generative Regularizers for Inverse Problems
Alexander Denker, Johannes Hertrich, Sebastian Neumayer
Generative (diffusion) priors demonstrate remarkable performance in addressing inverse problems in imaging. Yet, for scientific and medical imaging, it is crucial that reconstructi…
Numerical Methods for Kernel Slicing
Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer
Kernels are key in machine learning for modeling interactions. Unfortunately, brute-force computation of the related kernel sums scales quadratically with the number of samples. Re…
DeepInverse: A Python package for solving imaging inverse problems with deep learning
Julián Tachella, Matthieu Terris, Samuel Hurault +24
DeepInverse is an open-source PyTorch-based library for solving imaging inverse problems. The library covers all crucial steps in image reconstruction from the efficient implementa…
Generative Feature Training of Thin 2-Layer Networks
Johannes Hertrich, Sebastian Neumayer
We consider the approximation of functions by 2-layer neural networks with a small number of hidden weights based on the squared loss and small datasets. Due to the highly non-conv…
Stability of Data-Dependent Ridge-Regularization for Inverse Problems
Sebastian Neumayer, Fabian Altekrüger
Theoretical guarantees for the robust solution of inverse problems have important implications for applications. To achieve both guarantees and high reconstruction quality, we prop…