6 papers
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
Matthias Brändel, Stephan Köhler, Oliver Rheinbach
We study how the next-token prediction of an autoregressive Transformer language model changes under small perturbations of earlier input token embeddings. Motivated by operator le…
Overlapping Schwarz Attention: Hierarchical Attention via Domain Decomposition
Stephan Köhler, Oliver Rheinbach
We propose a hierarchical attention mechanism based on two-level overlapping Schwarz domain decomposition. The method is motivated by domain decomposition methods in partial differ…
Overlapping Schwarz Preconditioners for Pose-Graph SLAM in Robotics
Stephan Köhler, Oliver Rheinbach, Yue Xiang Tee +1
We investigate scalable two-level overlapping Schwarz domain decomposition methods with energy-minimizing coarse spaces of GDSW type (Generalized Dryja--Smith--Widlund type) as pre…
Four-Level Overlapping Schwarz as Multigrid Coarse Solver for Incompressible Non-Newtonian Flow in Complex Geometries
Stephan Köhler, Oliver Rheinbach
For complex geometries, the coarse problem of geometric multigrid can be too large to be solved by a direct solver. Here, we report on the use of domain decomposition applied to th…
Monolithic Multi-level Overlapping Schwarz Solvers for Fluid Problems
Stephan Köhler, Oliver Rheinbach
Additive overlapping Schwarz Methods are iterative methods of the domain decomposition type for the solution of partial differential equations. Numerical and parallel scalability o…
Combining Nonlinear FETI-DP Methods and Quasi-Newton Methods using an SQP Approach
Stephan Köhler, Oliver Rheinbach
The combination of nonlinear FETI-DP (Dual Primal Finite Element Tearing and Interconnecting) and Quasi-Newton methods using a sequential quadratic programming (SQP) approach is co…