4 papers
SympFormer: Accelerated attention blocks via Inertial Dynamics on Density Manifolds
Viktor Stein, Wuchen Li, Gabriele Steidl
Transformers owe much of their empirical success in natural language processing to the self-attention blocks. Recent perspectives interpret attention blocks as interacting particle…
Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions
Viktor Stein, Wuchen Li
Stein variational gradient descent (SVGD) is a kernel-based and non-parametric particle method for sampling from a target distribution, such as in Bayesian inference and other mach…
Expansive Natural Neural Gradient Flows for Energy Minimization
Wolfgang Dahmen, Wuchen Li, Yuankai Teng +1
This paper develops expansive gradient dynamics in deep neural network-induced mapping spaces. Specifically, we generate tools and concepts for minimizing a class of energy functio…
Accelerated Stein Variational Gradient Flow
Viktor Stein, Wuchen Li
Stein variational gradient descent (SVGD) is a kernel-based particle method for sampling from a target distribution, e.g., in generative modeling and Bayesian inference. SVGD does…