Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Quasi Linear Kernel Attention with Infinite Capacity
Nicolaj Rux, Johannes Hertrich, Sebastian Neumayer
The evaluation cost of transformers with softmax attention scales quadratically with sequence length. Kernel attention addresses this by replacing softmax with a more general kerne…
cs.LG2026
Fast Gauss Sums via Flash Attention
Nicolaj Rux, Sebastian Neumayer
Gaussian kernel sums are the computational core of maximum mean discrepancies (MMDs), kernel gradient flows, Stein variational gradient descent (SVGD), and many other kernel method…
cs.LG2024
Generative Feature Training of Thin 2-Layer Networks
Johannes Hertrich, Sebastian Neumayer
We consider the approximation of functions by 2-layer neural networks with a small number of hidden weights based on the squared loss and small datasets. Due to the highly non-conv…