1 citations · 2 across the 6 of their papers we have counts for
7 papers
Quasi Linear Kernel Attention with Infinite Capacity
Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer
The evaluation cost of transformers with softmax attention scales quadratically with sequence length. Kernel attention addresses this by replacing softmax with a more general kerne…
Fast Gauss Sums via Flash Attention
Nicolaj Rux, Sebastian Neumayer
Gaussian kernel sums are the computational core of maximum mean discrepancies (MMDs), kernel gradient flows, Stein variational gradient descent (SVGD), and many other kernel method…
Numerical Methods for Kernel Slicing
Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer
Kernels are key in machine learning for modeling interactions. Unfortunately, brute-force computation of the related kernel sums scales quadratically with the number of samples. Re…
Smoothed Distance Kernels for MMDs and Applications in Wasserstein Gradient Flows
Nicolaj Rux, Michael Quellmalz, Gabriele Steidl
Negative distance kernels were used in the definition of maximum mean discrepancies (MMDs) in statistics and lead to favorable numerical results in various ap…
Wasserstein Gradient Flows of MMD Functionals with Distance Kernels under Sobolev Regularization
Richard Duong, Nicolaj Rux, Viktor Stein +1
We consider Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals for positive and negative distance kernels and…
Slicing of Radial Functions: a Dimension Walk in the Fourier Space
Nicolaj Rux, Michael Quellmalz, Gabriele Steidl
Computations in high-dimensional spaces can often be realized only approximately, using a certain number of projections onto lower dimensional subspaces or sampling from distributi…
Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces
Viktor Stein, Sebastian Neumayer, Nicolaj Rux +1
Commonly used -divergences of measures, e.g., the Kullback-Leibler divergence, are subject to limitations regarding the support of the involved measures. A remedy is regularizin…